77 Commits

Author SHA1 Message Date
Steve Wilkerson
2840259fb8 Armada check: Add test directive to charts
This adds support for executing helm tests via the armada test
directive. It enables theses tests for all services, except for
nova and neutron as executing tests with armada force a chart to
wait. Forcing nova and neutron to wait effectively sequences the
charts, which will result in a failure to deploy past those
services

Depends-On: https://review.openstack.org/#/c/581148

Change-Id: I6ac845c82d744e2f5fd79c3e2ff3c1479dd1ddab
2018-07-11 13:34:58 +00:00
Steve Wilkerson
da7bc575ec Add logging.conf files to enabled loggers/handlers/formatters
This introduces a mechanism for generating the logging.conf
file for the openstack services via the values. This allows us to
define loggers, handlers, and formatters for the services and the
modules they're composed of.

This also allows us to take advantage of the oslo fluent handler
and formatter. The fluent handler and formatter give us the
following benefits: sending logs directly to fluentd instead of
routed to stdout/stderr and then through fluentbit to fluentd,
project specific tags on the logged events (enables us to define
more robust filters in fluentd for aggregation if required),
full traceback support, and additional metadata (modules that
created logged event, etc)

Depends-On: https://review.openstack.org/577796

Change-Id: I63340ce6b03191d93a74d9ac6947f0b49b8a1a39
2018-06-26 09:51:14 -05:00
Pete Birley
75ecf8fdf4 Gotpl: remove quote and trunc to suppress output
This PS removes the use of the `quote and truncate` approach to
suppress output from gotpl actions in templates and replaces it
with the recommended practice of defining `$_` instead.

Change-Id: I5f35c5f7e70b4f7f461d772e3b72ed1c695c56a8
Signed-off-by: Pete Birley <pete@port.direct>
2018-06-16 18:52:13 +00:00
Pete Birley
67aed694c4 Deployments: Use current kubernetes deployment version
This PS moves to use the current ga version for kubernetes deployments.

Story: 2002205
Task: 21735
Depends-On: Icb4e7aa2392da6867427a58926be2da6f424bd56

Change-Id: I062a8a29dff70427ee9bcf09f595011b3611b0b1
Signed-off-by: Pete Birley <pete@port.direct>
2018-06-13 21:29:59 -05:00
Pete Birley
ff2e807795 Mistral: add deps for rabbitmq job
This PS adds dep checking for the rabbitmq management jobs.

Change-Id: I513213317c235b57d6877dc5c012fef03f484c1b
Signed-off-by: Pete Birley <pete@port.direct>
2018-06-05 17:50:28 -05:00
Pete Birley
460675bf7f Add image management function to manifests
When removing helm-toolkit from OSH and swithcing to use the
toolkit from OSH-Infra, the image declaration function was missed.

Depends-On: I2f2012590d81ffcb159d49d8a76eedd4441744cd
Change-Id: I0f1118bb748f3fe1b6bb73acfc00e77c5cca9c7d
Signed-off-by: Pete Birley <pete@port.direct>
2018-05-20 10:10:16 -05:00
Steve Wilkerson
354b311ec5 Add local-registry image managment to OSH from OSH-Infra
This PS adds the local registry image managment to OSH from OSH-Infra.
With this the delta between helm-toolkits in the Repo's is removed,
allowing the toolkit from OSH-Infra to be used and the one from OSH
to be depreciated.

Change-Id: If5e218cf7df17261fe5ef249d281f9d9637e2f6a
Co-Authored-By: Pete Birley <pete@port.direct>
2018-05-12 14:35:48 +00:00
Pete Birley
356fa9e922 Mistral: Add helm test
This PS adds helm test functionaility to mistral.

Change-Id: I463d16d6e6fcd00afdc23365b58a8c1e1d395388
2018-05-04 02:34:44 +00:00
Pete Birley
39c65a9611 Mistral: Fix endpoint creation
This PS fixes the mistral endpoint creation to allow the service to
operate.

Change-Id: I6228809286cff55b23584b357a1e3d2caf8d22fb
2018-05-03 17:28:32 -05:00
Zuul
32a468178b Merge "Ingress controller service: consolidate to helm-toolkit" 2018-03-14 02:14:22 +00:00
Pete Birley
507600e898 Ingress controller service: consolidate to helm-toolkit
This PS consolidates the Ingress controller service, that is used
to resolve internal requests to public endpoints correctly, to
helm-toolkit.

Change-Id: If7c7deca1b8289a32709f7dc7c936883469aadfe
2018-03-12 13:48:39 +00:00
Pete Birley
6e4bcebcf5 DB-Drop-Jobs: consolidate to helm-toolkit
This PS consolidates the DB-Drop Job to helm-toolkit.

Change-Id: Ia2b035d730bf612086a9fd9b5d14aba494f56dc7
2018-03-09 14:25:15 +00:00
Zuul
4884dd228c Merge "Keystone Authtoken Cache: allow universal secret key to be set" 2018-03-05 19:26:37 +00:00
portdirect
1c85fdc390 Keystone Authtoken Cache: allow universal secret key to be set
This PS allows a cache secret key for all instances of keystone
middleware to be defined in a single location.

Change-Id: I3d5c78732d8a8bb9110117130f0d886fea609526
Partial-Bug: 1753251
2018-03-05 08:49:24 -05:00
Pete Birley
02136ea2f1 Mistral: use endpoints section and lookups to set port
This PS moves mistral to use the endpoints section and lookups to
set the port it serves on.

Change-Id: I3cbe2d5359a616595c2145fd23d6be65fcc77b3f
2018-03-03 18:23:09 +00:00
Renis
1ecc905b65 Rabbitmq Credential Management
- This PS implements job to create new user
for each chart

Change-Id: I7335ba4ad4bc9f70871100dbd9e6f030049abe07
2018-02-26 17:44:44 -08:00
Pete Birley
f57972b5b6 dependencies: move static dependencies under a 'static' key
This PS moves static dependencies under a 'static' key to allow
expansion to cover dynamic dependencies.

Change-Id: I38990b93aa79fa1f70af6f2c78e5e5c61c63f32c
2018-02-23 12:31:15 -08:00
Pete Birley
49aacc2030 Ingress rules: consolidate to helm-toolkit
This PS consolidates ingress rules to helm-toolkit.

Change-Id: I38a4de939e1ec65fed1630a53787d363f2ec78f6
2018-02-21 10:21:16 -08:00
Chris Wedgwood
6b844382ad yaml cleanup: trim multiline strings
Change-Id: Ice615c1d252651793dfa09b8e85a5b4228d68737
2018-02-20 16:39:52 +00:00
Zuul
e4313c86c7 Merge "remove unnecessary calls to quote" 2018-02-20 14:53:21 +00:00
portdirect
c7e2eb9e25 Bootstrap jobs: move template to helm toolkit
This PS moves the templates for bootstrap jobs to helm-toolkit.

Change-Id: I0fc0f7722cfc87b00e26510dee7ba79d2139a171
2018-02-19 22:53:34 -05:00
Chris Wedgwood
4f6a9983f5 remove unnecessary calls to quote
Change-Id: I1099c3f56b1d81b97033a8653d6bf1b9ea4d04e7
2018-02-19 18:55:46 +00:00
Pete Birley
b311f86193 Node Labels: update nodelabels to allow targeting of pods to nodes
This PS updates the node labels to allow pods to be targeted to nodes
on a per type basis.

Change-Id: I45d5383d04fcd1d98740a18d86c1cfc2cb8ec409
2018-02-19 11:51:09 -05:00
portdirect
eb943b63fb DB-Sync-Jobs: consolidate to helm-toolkit
This PS consolidates the DB-Sync Job to helm-toolkit.

Change-Id: I54d53468a437f6cacf6943ed3dec27089bf5f482
2018-02-18 21:08:24 -06:00
portdirect
897edb3202 DB-Init-Jobs: consolidate to helm-toolkit
This PS consolidates the DB-Init Job to helm-toolkit.

Change-Id: Ib92743d678de09a6fb4457e5415a098013952410
2018-02-17 22:47:58 +00:00
portdirect
8d30e6698e Helm-Toolkit: move keystone endpoint jobs to helm-toolkit
This PS moves the keystone endpoint jobs to be driven by
a helm-toolkit function providing greater consistency
to the charts in OSH, and reduced tech debt.

Change-Id: I06555171ce93a2b0a8798bc891a2990ea9626276
2018-02-07 19:51:41 -05:00
portdirect
914ffdbfe4 Helm-Toolkit: move keystone service jobs to helm-toolkit
This PS moves the keystone service jobs to be driven by
a helm-toolkit function providing greater consistency
to the charts in OSH, and reduced tech debt.

Change-Id: I36e012854473fcb942d62391d182ee6a8406bb72
2018-02-08 00:31:52 +00:00
portdirect
72792a49d4 Keystone: update user jobs to use defaults in dict keys
This PS updates the Keystone user job template to use default
values when not specified via the input dict.

Change-Id: I937e720ab3ab4ba70c33ea981d7693ebfd01c894
2018-02-07 11:03:17 -05:00
portdirect
319fd2cbec Jobs: Apply uniform node selector label to all jobs in OSH
This PS applies a uniform node selector label to all jobs in OSH.

Change-Id: I2efedcea2085b8189bd0bd12dd0ee66e63db0d97
2018-02-07 10:16:06 -05:00
portdirect
f296acf647 Helm-Toolkit: move keystone user jobs to helm-toolkit
This PS moves the keystone user jobs to be driven by
a helm-toolkit function providing greater consistency
to the charts in OSH, and reduced tech debt.

Change-Id: Ic5eb172b0443f61b8ecab8b3a607c764fb145c75
2018-02-07 09:49:04 -05:00
portdirect
b180d28618 Auth: Update credential keys to reference service specifically
This PS moves all credentials for OpenStack services from 'user' to
the service name. This allows a single yaml snippet to articulate
the credentials for a deployment.

Change-Id: Ic720109f2ba854561b23767cb480bcae91f74b6b
2018-01-15 18:54:13 +00:00
portdirect
5a2f71ebdf Ingress: Allow annotations to be dyanmicly driven
This PS allows the ingress rules to be dynamicly driven from the
values.yaml, permitting the ingress cotnroller to ba changed and
custom rules to be applied: eg whitelisting of clients.

Change-Id: Ica6b4692ff9b6b77d1efe6bae212a1227e56ca66
2018-01-05 00:29:05 -05:00
portdirect
fa2620d54b RBAC for OSH
This PS applys RBAC rules to OSH, based off the work
done in https://review.openstack.org/#/c/526464/

Change-Id: I541b0ac1a3972566ef2b66571ae32744dab70c17
2017-12-26 10:24:19 -05:00
Tin Lam
197b79592d Images: Mistral service specific explicit image names
This patch set makes the service-specific images for mistral
have explicit names, allowing simple over-riding of images
for an entire site.

Change-Id: I49a105a29660bbcc500e31352d639389f5636ed5
2017-12-03 05:28:17 +00:00
intlabs
fe6107cf76 Images: Update values to allow simple parse of images being used
This PS updates the values file layout for images to allow simple
parsing of the images in use by charts, allowing them to be queried
and modified much more simply. By moving the image tags to a 'tags'
key, we can extend the options used simply to accomodate extra
options simply (eg prefixing the tag for use with an internal
registry) or pre-pulling the images to reduce chart deploy failure.

Change-Id: I9ec1dbb00d997ab6cb021bf0b698f7aae740e95d
2017-10-23 10:05:20 -05:00
Sean Larimore
d1e775547c Adding database drop jobs
Implemented Kubernetes jobs to drop a database for an OpenStack service
on deletion of Helm chart. This job is configurable to be enabled or
disabled in the values.yaml file for each chart.

Change-Id: Ia1a0834cb43c7e883aaf507a7c7683fa1cf4e838
2017-09-27 09:44:28 -06:00
Pete Birley
c27aaec303 Mistral: remove oslo-genconfig
This PS removes the modified oslo-genconfig from mistral.

Partially implements: blueprint remove-pregenerated-config-templates

Change-Id: Iddcb0a90e761181867687e548d5e5daa1039d7c8
2017-09-22 14:42:30 -05:00
Mateusz Blaszkowski
e4f0690eb2 'affinity' constraints added in remaining services
Change-Id: I1ca654fbcc0301591e6b290d41fce1797ea2820a
Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>
2017-09-04 05:56:19 -04:00
Pete Birley
bede94c835 Endpoint lookup namespace and fqdn support
This PS adds namespace and fqdn support to endpoint lookup functions,
it also permits over-riding of the puplic endpoint for ingress.

Change-Id: Ib61c5c00a214d75fe85fbffe9080c2ae88bd8cb9
2017-08-31 19:06:33 +00:00
Pete Birley
46f9dea2c8 Security: Add Pod user to missed services
This PS adds the pod user to missed services, which is required
for fully agnosic image operation.

Change-Id: I09c481f90b5e6a478e699ebaae51a2316ceb6a8d
2017-08-28 20:19:22 -05:00
Pete Birley
7cfd182929 Security: Make policy fully configurable via helm values
This PS moves the policy.json to be fully driven by gotpl,
allowing full configuration without editing the template.

Nova and Cinder are addressed in the seperate patchsets:
 * https://review.openstack.org/#/c/498215/
 * https://review.openstack.org/#/c/498216/

Change-Id: Ia2be5fb4e460d41034fdadbbefc1e48d0869e023
2017-08-26 17:50:22 -05:00
Jenkins
85b6716c49 Merge "Fix for appending custom volumes" 2017-08-25 15:00:10 +00:00
Jenkins
d04e69c438 Merge "Enable keystonemiddleware cache encryption" 2017-08-25 01:56:10 +00:00
Mateusz Blaszkowski
fc9677f718 Fix for appending custom volumes
Before this change, there was no ability to append custom volume for
any of the services. The reason was a missing new line character, so
the templates were formatted like this:
- name: pod-shared
  mountPath: /tmp/pod-shared - mountPath: /tmp/test2
  name: test2

Apart from that, for some of the services (mostly job-bootstrap) invalid
indentation for custom volumes (and their mounts) was set.

Closes-Bug: 1712745
Change-Id: Ib57c76a34c0e28ad9e67ea47d1fc250b17711a42
Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>
2017-08-24 06:24:47 -04:00
Pete Birley
1fd7f66783 Helm-Toolkit: Configmap templater
This PS adds a configmap teplater helper to helm-toolkit. It makes it
simpler to write consistent charts that supports over-riding of all
values.

Change-Id: I9a587999859ea02802485eb25a3f0ebec8c712a8
2017-08-23 08:06:38 -05:00
Tin Lam
c35f08c4fa Enable keystonemiddleware cache encryption
This patch set enables keystonemiddleware memcache encryption by providing a
random string key into the service configuration file, and setting the
memcache_security_strategy as ENCRYPT.

Change-Id: Ia030f5414308a29096c644bae70047a323eaffde
2017-08-15 22:47:55 -05:00
Pete Birley
ff8744a9b9 Licence: Remove licence header from rendered files
This PS removes the licence header from rendered output from tiller,
significantly reducing the configmap size of charts deployed to the
cluster.

Change-Id: I5d1b246f2068f3b83bf59ba79fe8b88bbc9a6161
2017-08-07 17:16:13 -05:00
Pete Birley
35601e2bd3 Charts: make manifests optional for all OpenStack elements
This PS allows the rendering of manifests to be controlled. It enables
both increased control over deployment when required but also makes
development of a feature easier to target.

Change-Id: I1716e8ee23fe5c53f935bd739ea283bc4a2a9963
2017-08-07 14:59:48 -05:00
Pete Birley
8f8a80019a Jobs: Add labels to all jobs
This PS adds labels to all jobs in OpenStack-Helm, allowing them to be
found by label searches. This makes management of large clusters using
tools like Armada easier.

Change-Id: I49b2cb7a94fab96958f187ca11e2c2a0c80ff843
2017-08-05 20:17:12 +00:00
Pete Birley
cbae720d1d Values: Move mounts to pod key
This PS moves the mounts key to be under the pod key in the values.
It brings further consolation of related configuration params to be
nested under common keys across all charts.

Change-Id: If9963e4f8b438847e2fcad3bdd8c0d71ca9ecdd8
2017-07-14 10:44:35 -05:00