89 Commits

Author SHA1 Message Date
Pete Birley
16a250509a Telemetry: enable publishing of events for core OpenStack Components
This PS enables publishing of events for core OpenStack Components

Change-Id: I846bfa86ae522c103e741b46e189695d45ca7710
2017-10-04 17:19:45 +00:00
Sean Larimore
d1e775547c Adding database drop jobs
Implemented Kubernetes jobs to drop a database for an OpenStack service
on deletion of Helm chart. This job is configurable to be enabled or
disabled in the values.yaml file for each chart.

Change-Id: Ia1a0834cb43c7e883aaf507a7c7683fa1cf4e838
2017-09-27 09:44:28 -06:00
Pete Birley
6df59fc3d9 Heat: remove oslo-genconfig
This PS removes the modified oslo-genconfig from heat.

Partially implements: blueprint remove-pregenerated-config-templates

Change-Id: Ib4a5e1c41874e3a2eb15b5002538a2193f07ab04
2017-09-21 13:50:09 +00:00
Jenkins
15dc95754d Merge "Adding missing labels and mounts to heat-trusts job" 2017-09-06 18:55:55 +00:00
Pete Birley
b120667031 Endpoint lookups: fix typo in overrides
This PS fixes a typo in the endpoint lookup function and input.

Change-Id: I6ff6d68bf82e1e1f70438a7017f4e1a6df7f59a7
2017-09-06 11:15:54 -05:00
Mateusz Blaszkowski
6e2ae0b739 Adding missing labels and mounts to heat-trusts job
Change-Id: I19c85574f78d2c90019ad1f94cba45630c8f48de
Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>
2017-09-04 04:38:09 -04:00
Pete Birley
bede94c835 Endpoint lookup namespace and fqdn support
This PS adds namespace and fqdn support to endpoint lookup functions,
it also permits over-riding of the puplic endpoint for ingress.

Change-Id: Ib61c5c00a214d75fe85fbffe9080c2ae88bd8cb9
2017-08-31 19:06:33 +00:00
Pete Birley
ceb30e8cc7 Jobs: Consoloate on heat-engine for admin jobs where possible.
This ps moves to use a container sultaible for use as the heat engine
for all possible admin jobs - it is lighter than the kolla-toolbox image
and makes it easy to swap out to other image sets. This is as the heat
engine container should contain the openstack client (with all required
libs for the cloud) and the oslo_db supporting libs required by the db
management jobs, as well as the oslo_messaging libs required for future
rabbitmq management expansion.

Change-Id: I5451c15c8fb49c85b4f254cc60156420bee2efea
2017-08-29 04:34:26 +00:00
Pete Birley
7cfd182929 Security: Make policy fully configurable via helm values
This PS moves the policy.json to be fully driven by gotpl,
allowing full configuration without editing the template.

Nova and Cinder are addressed in the seperate patchsets:
 * https://review.openstack.org/#/c/498215/
 * https://review.openstack.org/#/c/498216/

Change-Id: Ia2be5fb4e460d41034fdadbbefc1e48d0869e023
2017-08-26 17:50:22 -05:00
Pete Birley
27864cec04 Security: Container user for support openstack services
This PS adds user control for the service user for support openstack
services.

Change-Id: I132511bfc09d20a2f155bb9498a7fe8eeee8b6f9
2017-08-26 22:03:15 +00:00
Jenkins
85b6716c49 Merge "Fix for appending custom volumes" 2017-08-25 15:00:10 +00:00
Jenkins
d04e69c438 Merge "Enable keystonemiddleware cache encryption" 2017-08-25 01:56:10 +00:00
Mateusz Blaszkowski
fc9677f718 Fix for appending custom volumes
Before this change, there was no ability to append custom volume for
any of the services. The reason was a missing new line character, so
the templates were formatted like this:
- name: pod-shared
  mountPath: /tmp/pod-shared - mountPath: /tmp/test2
  name: test2

Apart from that, for some of the services (mostly job-bootstrap) invalid
indentation for custom volumes (and their mounts) was set.

Closes-Bug: 1712745
Change-Id: Ib57c76a34c0e28ad9e67ea47d1fc250b17711a42
Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>
2017-08-24 06:24:47 -04:00
Pete Birley
1fd7f66783 Helm-Toolkit: Configmap templater
This PS adds a configmap teplater helper to helm-toolkit. It makes it
simpler to write consistent charts that supports over-riding of all
values.

Change-Id: I9a587999859ea02802485eb25a3f0ebec8c712a8
2017-08-23 08:06:38 -05:00
Michał Dulko
da137ac70d Fix issues with [trustee] section of heat.conf
There are serveral issues with default settings of [trustee] section in
heat.conf:

1. Keystone trust isn't added for admin user (heat-trust should be
   admin's user trustee to make it possible for admin to create stacks).
2. Keystone is adding role "admin" in domain "heat". This blocks
   creation of correct trust in Keystone as role names are duplicated.
   Please note that adding this role is not necessary for Heat to work
   correctly.

This commit solves the issues by:

1. Creating a job that will add a Keystone trust between admin and
   heat-trust users. This allows admin to create Heat stacks.
2. Removes adding a new role in a domain in _ks-domain-user.sh.tpl
   script.

Additionally, as _ks-domain-user.sh.tpl is only really used by Heat
chart, this commit also removes it from configmap-bin in Barbican,
Magnum, Mistral and Senlin charts. Those charts must have been
copy-pasted from Heat chart and don't need to include this file.

Also I fix a bug introduced by I86a21e625afd822379ac11351603b2c606a3769f
that renamded heat-domain user to heat-trust and created two users with
the same name.

Change-Id: I303d9bc2aa1796f21bedc6ecdc85a4b3f6c68504
Closes-Bug: 1696462
2017-08-21 20:10:11 +00:00
Tin Lam
c35f08c4fa Enable keystonemiddleware cache encryption
This patch set enables keystonemiddleware memcache encryption by providing a
random string key into the service configuration file, and setting the
memcache_security_strategy as ENCRYPT.

Change-Id: Ia030f5414308a29096c644bae70047a323eaffde
2017-08-15 22:47:55 -05:00
Pete Birley
ff8744a9b9 Licence: Remove licence header from rendered files
This PS removes the licence header from rendered output from tiller,
significantly reducing the configmap size of charts deployed to the
cluster.

Change-Id: I5d1b246f2068f3b83bf59ba79fe8b88bbc9a6161
2017-08-07 17:16:13 -05:00
Pete Birley
35601e2bd3 Charts: make manifests optional for all OpenStack elements
This PS allows the rendering of manifests to be controlled. It enables
both increased control over deployment when required but also makes
development of a feature easier to target.

Change-Id: I1716e8ee23fe5c53f935bd739ea283bc4a2a9963
2017-08-07 14:59:48 -05:00
Pete Birley
8f8a80019a Jobs: Add labels to all jobs
This PS adds labels to all jobs in OpenStack-Helm, allowing them to be
found by label searches. This makes management of large clusters using
tools like Armada easier.

Change-Id: I49b2cb7a94fab96958f187ca11e2c2a0c80ff843
2017-08-05 20:17:12 +00:00
Pete Birley
f30cbcf108 Update pod affinity to allow customisation
This PS updates the pod affinity function to allow customisation by
operators at the point of deployment.

Change-Id: I8b7b2f584e990e068051d9a6d5cc7b1e1adb5aa5
2017-07-28 15:02:47 +00:00
Pete Birley
cbae720d1d Values: Move mounts to pod key
This PS moves the mounts key to be under the pod key in the values.
It brings further consolation of related configuration params to be
nested under common keys across all charts.

Change-Id: If9963e4f8b438847e2fcad3bdd8c0d71ca9ecdd8
2017-07-14 10:44:35 -05:00
Pete Birley
4d066332a7 Values: Move replicas to pod key
This PS move s the replicas key to be under the pod key in the values.
It brings further consolation of related configuration params to be
nested under common keys across all charts.

Change-Id: I420b06debd0a62ba5d83497be43ff6c49c49d339
2017-07-13 23:08:08 -05:00
Pete Birley
3fe1fc3354 Apply bootstrap framework to all remaining services
This PS applies the bootstrap framework to all remaining services.

Change-Id: Ib77778d69e743b1594fbadaf9842e95541703bd1
External-Tracking-ID: OSH-36
2017-07-12 08:09:11 -05:00
Pete Birley
20527cb57c Move k8s pod params under common key
This PS moves K8s pod params under a common key.

Change-Id: Ib4e72e82a4f1540fa800c39d0dcc91b286d96c93
2017-07-11 23:28:47 +00:00
Tin Lam
670a78bcbe Fix file permission and readOnly flag
This patchset enforces stricter file permission on *-etc configmap and
sets readOnly flag to true in a number of charts.

Change-Id: I233689a5d56dd1352e0d81997a94b4cdd6bed5d2
Signed-off-by: Tin Lam <tin@irrational.io>
2017-07-08 13:24:14 -05:00
Pete Birley
ac06f49849 Unify and normalise K8s Resources and Upgrade Strategy
This PS unifies and normalises Kubernetes resource allocation and
update strategy across all OpenStack-Helm elements.

Change-Id: Ia41fc453cb5191fa447ca6e1aa0f5b431c939dc8
2017-07-06 17:48:16 +00:00
Pete Birley
c221fa5248 Secret and endpoint standardisation.
This PS moves keystone credentials to the endpoints section within
the values.yaml, and also adds a 'secrets' key, allowing standardiation
of secrets and credential management across OpenStack-Helm.

Change-Id: I86a21e625afd822379ac11351603b2c606a3769f
2017-07-05 12:22:04 -05:00
Jenkins
2d7fe938c0 Merge "Fix MultiOpt options in gen-oslo-openstack-helm" 2017-07-05 13:46:14 +00:00
Michał Dulko
9eaa08678c Fix MultiOpt options in gen-oslo-openstack-helm
gen-oslo-openstack-helm generated configuration file templates in
incorrect form, causing setting mulitple values in Values.yaml to
produce something like:

  foo=barfoo=baz

This commit fixes this in the generator and updates config file
templates to generate configs correctly:

  foo=bar
  foo=baz

Change-Id: Iea661dcf1710987b2e111d7141ba888f01c44a50
Closes-Bug: 1699581
2017-07-04 17:26:06 +02:00
Pete Birley
c5bbc97ab7 Soft pod anti-affinity for all pods
This PS adds soft anti-affinity to all pods in OS-H. By doing so
resiliancy is improved by attempting to ensure that pods are created
on seperate nodes.

Change-Id: I0c1092498f7a1e44218ef785ca3f73fa9f49819c
2017-07-03 14:15:40 -05:00
Pete Birley
c0cf2337a2 Ingress controller for public endpoints
This PS provides the framework for Ingress controller based access to
public APIs.

Services covered:
 * Barbican
 * Cinder
 * Glance
 * Heat
 * Horizon
 * Keystone
 * Magnum
 * Mistral
 * Neutron
 * Nova
 * Senlin

Change-Id: I82d55f6ca41859ebed3312ec7d95b353df09e772
Implements: blueprint api-ingress
External-Tracking-Id: OSH-58, OSH-62
2017-06-28 17:37:10 -05:00
Pete Birley
198ce69e71 Keystone: Member role fixes
This PS makes the initial Keystone Admin and Service account users
members of the projects that they are in.

Change-Id: I2de081a216b7e7b99d301ab605c84d882cdb5840
2017-06-24 16:28:56 +00:00
Pete Birley
a7503a7d3f Kubernetes Entrypoint: Lookup service names for dependency checks
The existing entrypoint logic used static names to reolve dependencies.
This prevented the service names, and thus the hostnames of services
being altered. This PS resolves that issue by looking up the service name
from the endpoints specified in the values for a chart.

Partial-Implements: blueprint enhance-entrypoint-dependency-checking
External-Tracking-Id: OSH-21
Change-Id: Ib49490f332f8cd88e98c50d9335dfd314a170936
2017-06-20 08:51:36 -05:00
Pete Birley
81aef1ecea Configmaps: set default mode of files in *-bin configmaps to +x
This PS sets the default modetype of mounts from *-bin configmaps
to 0555, and removes the then unnecessary commands from the manifests.

Change-Id: I93ce0facb06affdf362a58f8520e69ba94ea3034
2017-06-12 00:22:30 -05:00
Steve Wilkerson
161170bf96 Move init containers from annotation to official object
With 1.6, init containers are officially part of the kubernetes
API.  This changes the format of the helm template for the
entrypoint container from json to yaml, and updates the
charts accordingly.

Co-Authored-By: Pete Birley <pete@port.direct>

Change-Id: I569566ce4b031d107af2d38483040a26210bec45
2017-06-09 16:31:53 -05:00
Pete Birley
0df7427dc4 Refactor start scripts, to support graceful termination
This PS add Barbican support, and moves all potentially container
specific logic into the service start script from the api manifests.

Also fixes a permissions issue with the nova-api, which incorrectly
had the NET_ADMIN capability.

Change-Id: I18fc1ea5d7aa70ea7dabb829361a3da57e905100
2017-06-05 17:37:22 -05:00
Darla Ahlert
c394d22112 Add Graceful Termination to all existing charts
This commit adds graceful termination to all existing charts.
It also adds a setting in the values.yaml file for clarity and
the ability to override if wished.

Change-Id: I42025e4be86d248be467c1d2f0980f864c4d440e
2017-06-05 12:38:26 -05:00
Jenkins
592692fdf8 Merge "Update deployment for kubernetes 1.6" 2017-06-02 15:35:15 +00:00
Darla Ahlert
78dd37efc2 Update deployment for kubernetes 1.6
In Kubernetes 1.6, the deployment API version was changed
from extensions/v1beta1 to apps/v1beta1

Change-Id: I14ee465de381d25866224b9eae3512f0b42588d3
2017-06-02 08:45:08 -05:00
Pete Birley
e82846f1c7 Consistency: Move Heat container start commands to scripts
This PS moves the container start commands to scripts. This brings
the service into alignment with others in OpenStack-Helm, and enables
easier debugging and modifcation of the launch commands for services.

Change-Id: I9a3ca50ea303fe7e866f41f6acb0194cc2aa3dac
2017-06-01 10:34:42 -05:00
Jenkins
a5420f4290 Merge "Helm-Toolkit: Update naming to be explict and remove old functions." 2017-06-01 14:49:54 +00:00
Jenkins
3d88336677 Merge "Pod Disruption Budget: Set default to 0" 2017-05-31 21:28:39 +00:00
Pete Birley
8c29949c0d Helm-Toolkit: Update naming to be explict and remove old functions.
This PS updates the way helm-toolkit functions are named to
reference the full path they are loacted at. This should make
development and debugging easier. Addtionally unused functions
have been pruned as well.

Change-Id: I03c553f1d01bccc70c86768b416b147c90d9b2f0
2017-05-31 19:52:02 +00:00
Pete Birley
5e193580ed Pod Disruption Budget: Set default to 0
With pod disruption budget set it is not possible to drain nodes
and perform other expected operations. This PS changes the default
value to 0, restoring the typically expected operation of pods.

Change-Id: I43ef35638c98caee2cf567487173399157000ada
2017-05-31 06:11:58 -05:00
Mark Burnett
26517c908c Remove ConfigMap hash annotations from StatefulSets
Kubernetes does not allow updating annotations on StatefulSets (it
actually only allows a couple of fields to be modified), so the
existence of these hash annotations makes it impossible to modify
configmap values.

This will be a problem not only for upgrades, but for normal operation
with charts like `mariadb`, which will occasionally require
`helm upgrade ...` to modify settings to allow recovery from certain
cluster failure modes.

Change-Id: I5f0433fc05c5aa22b4ad7ec296efbde3acca3d5a
2017-05-30 15:42:52 -05:00
Mark Burnett
de20cbb15d Use the standard Dockerhub MariaDB image
Addresses consistency issues that appear with autoincrement fields in
the existing chart, as demonstrated in [1].  It should allow automatic
recovery of 2 out of the 3 (default) cluster members.

It does not address automatic recovery of a complete shutdown of all
cluster nodes.

[1] https://review.openstack.org/#/c/465977/

Change-Id: I84c86e1862f03a6d381bf219b821ea3636049f0b
2017-05-25 08:56:35 -05:00
Jenkins
ef6657deab Merge "Adding pod disruption budgets for heat" 2017-05-23 12:04:20 +00:00
Jenkins
fb11f693ab Merge "Cleanup - add remaining configmap hashes" 2017-05-20 23:08:07 +00:00
Larry Rensing
3d9b17cd8e Adding pod disruption budgets for heat
This PS adds a pod disruption budget for heat api, cfn, cloudwatch,
and engine to ensure 1 replica of each pod is running.

Change-Id: I8f5f224be7e804155390e18aa56dd86331f3645c
2017-05-19 18:14:21 +00:00
Larry Rensing
fea453ba72 Consolidate volumes across charts
This PS standardizes configmap mounts across all charts to be named
<project>-etc or <project>-bin.  This reduces the amount of volumes
listed in each file.

Change-Id: Idbaf495fa243bb9ea0f6ad755ca4f899d5fc0d6f
2017-05-16 15:49:51 +00:00