5087 Commits

Author SHA1 Message Date
Radosław Piliszek
03b4c706fa Allow cinder coordination backend to be configured
This is to allow operator to prevent enabling redis and/or
etcd from magically configuring cinder coordinator.

Note this change is backwards-compatible.

Change-Id: Ie10be55968e43e3b9cc347b1b58771c1f7b1b910
Related-Bug: #1840070
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-08-15 07:48:28 +00:00
Zuul
64d587b819 Merge "Fix swift log level configuration" 2019-08-13 17:03:47 +00:00
Scott Solkhon
dea87cde97 Fix swift log level configuration
Change-Id: I7f980640e75a9328a14a3e14e9c55358955f3182
2019-08-13 12:28:38 +00:00
Zuul
5c70e0a615 Merge "Set 'distro_python_version' variable" 2019-08-13 04:00:55 +00:00
Marcin Juszkiewicz
bf7ed6be04 Set 'distro_python_version' variable
We use that variable in Kolla in many places. There are places in
'kolla-ansible' where we also need it.

Change-Id: Iea78c4a7cb0fd1405ea7299cdcf0841f63820c8c
2019-08-12 13:23:42 +00:00
Zuul
b16bb0d787 Merge "Do not require EPEL repo on RHEL-based target hosts" 2019-08-10 00:33:53 +00:00
Zuul
4468250b95 Merge "Remove support for Docker legacy packages" 2019-08-09 15:27:09 +00:00
Zuul
3a37131f1d Merge "Fix FWaaS service provider (v2, Stein issue)" 2019-08-09 12:05:56 +00:00
Radosław Piliszek
85a5fb55c4 Fix FWaaS service provider (v2, Stein issue)
Because we merged both [1] and [2] in master,
we got broken FWaaS.
This patch unbreaks it and is required to backport
to Stein due to [2] backport waiting for merge,
while [1] is already backported.

[1] https://review.opendev.org/661704
[2] https://review.opendev.org/668406

Change-Id: I74427ce9b937c42393d86574614603bd788606af
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-08-08 20:26:57 +02:00
Doug Szumski
339ea2bdeb Support namespacing RabbitMQ logs
The RabbitMQ role supports namespacing the service via the
project_name. For example, if you change the project_name, the
container name and config directory will be renamed accordingly. However
the log folder is currently fixed, even though the service tries to
write to one named after the project_name. This change fixes that.

Whilst you might generally use vhosts, running multiple RabbitMQ
services on a single node is useful at the very least for testing,
or for running 'outward RabbitMQ' on the same node.

This change is part of the work to support Cells v2.

Partially Implements: blueprint support-nova-cells
Change-Id: Ied2c24c01571327ea532ba0aaf2fc5e89de8e1fb
2019-08-08 16:46:32 +00:00
Zuul
ee5e99fcf5 Merge "Stop using MountFlags=shared in Docker configuration" 2019-08-08 10:57:03 +00:00
Michal Nasiadka
ad9e8786a3 Add support for sha256 in ceph key distribution
- add support for sha256 in bslurp module
- change sha1 to sha256 in ceph-mon ansible role

Depends-On: https://review.opendev.org/655623
Change-Id: I25e28d150f2a8d4a7f87bb119d9fb1c46cfe926f
Closes-Bug: #1826327
2019-08-07 11:57:46 +00:00
Marcin Juszkiewicz
35941738d5 Stop using MountFlags=shared in Docker configuration
According to Docker upstream release notes [1] MountFlags should be
empty.

1. https://docs.docker.com/engine/release-notes/#18091

"Important notes about this release

In Docker versions prior to 18.09, containerd was managed by the Docker
engine daemon. In Docker Engine 18.09, containerd is managed by systemd.
Since containerd is managed by systemd, any custom configuration to the
docker.service systemd configuration which changes mount settings (for
example, MountFlags=slave) breaks interactions between the Docker Engine
daemon and containerd, and you will not be able to start containers.

Run the following command to get the current value of the MountFlags
property for the docker.service:

sudo systemctl show --property=MountFlags docker.service
MountFlags=

Update your configuration if this command prints a non-empty value for
MountFlags, and restart the docker service."

Closes-bug: #1833835

Change-Id: I4f4cbb09df752d00073a606463c62f0a6ca6c067
2019-08-07 13:50:46 +02:00
Mark Goddard
f63e36780b Remove support for Docker legacy packages
Docker is now always installed using the community edition (CE)
packages.

Change-Id: I8c3fe44fd9d2da99b5bb1c0ec3472d7e1b5fb295
2019-08-06 18:34:19 +01:00
Zuul
3731da0b79 Merge "Add mon address to ceph release version check" 2019-08-06 17:04:13 +00:00
Zuul
9a652b29e5 Merge "Support mon and osd to be named with hostname" 2019-08-06 13:59:08 +00:00
Zuul
418e9cccc7 Merge "ceph: fixes to deployment and upgrade" 2019-08-06 13:59:06 +00:00
Zuul
ca1de25fbf Merge "Add Kafka input to telegraf config" 2019-08-05 10:58:05 +00:00
Zuul
5760cc226b Merge "Fix checking mongodb replication status" 2019-08-05 09:02:05 +00:00
Zuul
8f70bc22d6 Merge "Add extra volumes support for services that were not previously supported" 2019-08-05 09:02:04 +00:00
wangwei
cd519db139 Support mon and osd to be named with hostname
In the current deployment of ceph, the node name of osd and the name
of mon are both IP, and other daemons use hostname.

This commit adds support for naming mon and osd nodes using hostname,
and does not change the default ip-named way.

Change-Id: I22bef72dcd8fc8bcd391ae30e4643520250fd556
2019-08-05 08:54:01 +00:00
Zuul
daba362f43 Merge "Handle more return codes from nova-status upgrade check" 2019-08-05 08:42:10 +00:00
Zuul
8615adefbc Merge "[gnocchi] Don't recursively modify file perms on start" 2019-08-05 08:42:08 +00:00
pangliye
93e868360d Add Kafka input to telegraf config
Change-Id: I9a8d3dc5f311d4ea4e5d9b03d522632abc66a7ac
2019-08-05 07:26:46 +00:00
Radosław Piliszek
67cedb7ad5 Do not require EPEL repo on RHEL-based target hosts
This change makes kolla-ansible more compatible with
RHEL which does not provide epel-release package.

EPEL was required to install simplejson from rpm
which was an ansible requirement when used python
version was below 2.5 ([1]). This has been obsolete for
quite a time so it's a good idea to get rid of it.

This change includes update of docs to read more properly.

[1] https://docs.ansible.com/ansible/2.3/intro_installation.html

Change-Id: I825431d41fbceb824baff27130d64dabe4475d33
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-08-05 09:01:49 +02:00
Radosław Piliszek
826f6850d0 ceph: fixes to deployment and upgrade
1) ceph-nfs (ganesha-ceph) - use NFSv4 only
This is recommended upstream.
v3 and UDP require portmapper (aka rpcbind) which we
do not want, except where Ubuntu ganesha version (2.6)
forces it by requiring enabled UDP, see [1].
The issue has been fixed in 2.8, included in CentOS.
Additionally disable v3 helper protocols and kerberos
to avoid meaningless warnings.

2) ceph-nfs (ganesha-ceph) - do not export host dbus
It is not in use. This avoids the temptation to try
handling it on host.

3) Properly handle ceph services deploy and upgrade
Upgrade runs deploy.
The order has been corrected - nfs goes after mds.
Additionally upgrade takes care of rgw for keystone
(for swift emulation).

4) Enhance ceph keyring module with error detection
Now it does not blindly try to create a keyring after
any failure. This used to hide real issue.

5) Retry ceph admin keyring update until cluster works
Reordering deployment caused issue with ceph cluster not being
fully operational before taking actions on it.

6) CI: Remove osd df from collected logs as it may hang CI
Hangs are caused by healthy MON and no healthy MGR.
A descriptive note is left in its place.

7) CI: Add 5s timeout to ceph informational commands
This decreases the timeout from the default 300s.

[1] https://review.opendev.org/669315

Change-Id: I1cf0ad10b80552f503898e723f0c4bd00a38f143
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-08-05 06:26:25 +00:00
Zuul
6ef646856f Merge "Remove unnecessary option from group_vars/all.yml" 2019-08-03 18:09:03 +00:00
Zuul
b59791ca92 Merge "Fix handling of docker restart policy" 2019-08-03 16:27:46 +00:00
chenxing
a1ab06d244 Remove unnecessary option from group_vars/all.yml
We often specific the project name after "{{ node_config_directory }}",
for example,
``{{ node_config_directory }}/cinder-api/:{{ container_config_directory }}/:ro``.
As the  "{{ project }}" option is not configured, This line was
generated with:
``/etc/kolla//cinder-api/:...``
There would be double slash exists. It's OK, but confusing.

Change-Id: I82e6a91b2c541e38cf8e97896842149b31244688
Closes-Bug: #1838259
2019-08-02 09:53:45 +08:00
Radosław Piliszek
19b345de3e Fix fluentd monasca pos path for Debian/Ubuntu x86_64
Change-Id: I6d205fe327f198e699519ebe9d589b9ee77a62d2
Closes-Bug: #1837274
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-07-30 12:43:20 +02:00
Zuul
2b78c831fd Merge "Remove FWaaS v1 related options" 2019-07-23 12:16:50 +00:00
Jeffrey Zhang
1537f6ed66 Remove FWaaS v1 related options
Neutron FWaaS v1 is deprecated and removed since stein cycle by [0]. So
remove related options in kolla.

[0] https://review.opendev.org/616410

Change-Id: Ia03e7979dd48bafb34c11edd08c2a2a87b949e0e
2019-07-23 08:24:40 +08:00
Zuul
249c6a2624 Merge "Fix glance bootstrap with file backend" 2019-07-22 08:04:33 +00:00
Jason
7d28476177
Do not recreate Blazar DB if using preconfigured
Most other services already gate the DB bootstrap operations with the
'use_preconfigured_databases' variable; Blazar did not.

Change-Id: I772b1cb92612c7e6936f052ed9947f93582f264c
2019-07-18 17:22:45 -05:00
Jason Anderson
464fefb1ef
[gnocchi] Don't recursively modify file perms on start
For deployments with a lot of Gnocchi data, this is a non-starter
(literally... the service basically can't start.) There maybe needs to
be a way to configure this, or only do it during deploy/bootstrap?
Unclear, but disabling for now; users can `chown -R gnocchi:gnocchi`
themselves in the meantime if need be.

Change-Id: I0bae6dfbbee9f63506c89bd6b392e7be07fd5930
2019-07-18 17:18:46 -05:00
Mark Goddard
1abd15d4a6 Fix glance bootstrap with file backend
Change https://review.opendev.org/#/c/670247/ attempted to fix glance
deployment with the file backend. However, it added a new bug by being
more strict about only generating configuration where the container will
be deployed. This means that the current method of running the glance
bootstrap container on any host in glance-api group could be broken,
since it needs the container configuration.

This change only runs the bootstrap container on hosts in the
glance_api_hosts list, which in the case of the file backend typically
only contains one host.

This change also fixes up some logic during rolling upgrade, where we
might not generate new configuration for the bootstrap host.

Change-Id: I83547cd83b06ddefb3a9e1f39844537bdb32bd7f
Related-Bug: #1836151
2019-07-18 14:36:56 +00:00
Radosław Piliszek
6a737b1968 Fix handling of docker restart policy
Docker has no restart policy named 'never'. It has 'no'.
This has bitten us already (see [1]) and might bite us again whenever
we want to change the restart policy to 'no'.

This patch makes our docker integration honor all valid restart policies
and only valid restart policies.
All relevant docker restart policy usages are patched as well.

I added some FIXMEs around which are relevant to kolla-ansible docker
integration. They are not fixed in here to not alter behavior.

[1] https://review.opendev.org/667363

Change-Id: I1c9764fb9bbda08a71186091aced67433ad4e3d6
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-07-18 13:39:06 +00:00
Zuul
b7098faf88 Merge "Fix ironic inspector iPXE boot with UEFI" 2019-07-17 20:06:51 +00:00
Zuul
f52eee42c2 Merge "ceph-nfs: Add rpcbind to Ubuntu host bootstrap" 2019-07-17 16:02:43 +00:00
Michal Nasiadka
efcaf400b8 ceph-nfs: Add rpcbind to Ubuntu host bootstrap
* Ubuntu ships with nfs-ganesha 2.6.0, which requires to do an rpcbind
udp test on startup (was fixed later)
* Add rpcbind package to be installed by kolla-ansible bootstrap when
ceph_nfs is enabled
* Update Ceph deployment docs with a note

Change-Id: Ic19264191a0ed418fa959fdc122cef543446fbe5
2019-07-16 17:38:18 +00:00
Zuul
61a74c0f5b Merge "Do not require valid migration_interface for controllers" 2019-07-16 14:10:09 +00:00
Zuul
0f67261352 Merge "Enhance merge_* action plugins to allow expected relative includes" 2019-07-16 14:10:07 +00:00
Zuul
dc1bce370e Merge "Fix glance with file backend" 2019-07-14 12:17:46 +00:00
Mark Goddard
7b9397566a Fix ironic inspector iPXE boot with UEFI
The ironic inspector iPXE configuration includes the following kernel
argument:

initrd=agent.ramdisk

However, the ramdisk is actually called ironic-agent.initramfs, so the
argument should be:

initrd=ironic-agent.initramfs

In BIOS boot mode this does not cause a problem, but for compute nodes
with UEFI enabled, it seems to be more strict about this, and fails to
boot.

Change-Id: Ic84f3b79fdd3cd1730ca2fb79c11c7a4e4d824de
Closes-Bug: #1836375
2019-07-12 15:09:56 +01:00
Mark Goddard
d5e5e885d1 During deploy, always sync DB
A common class of problems goes like this:

* kolla-ansible deploy
* Hit a problem, often in ansible/roles/*/tasks/bootstrap.yml
* Re-run kolla-ansible deploy
* Service fails to start

This happens because the DB is created during the first run, but for some
reason we fail before performing the DB sync. This means that on the second run
we don't include ansible/roles/*/tasks/bootstrap_service.yml because the DB
already exists, and therefore still don't perform the DB sync. However this
time, the command may complete without apparent error.

We should be less careful about when we perform the DB sync, and do it whenever
it is necessary. There is an argument for not doing the sync during a
'reconfigure' command, although we will not change that here.

This change only always performs the DB sync during 'deploy' and
'reconfigure' commands.

Change-Id: I82d30f3fcf325a3fdff3c59f19a1f88055b566cc
Closes-Bug: #1823766
Closes-Bug: #1797814
2019-07-12 08:56:54 +00:00
Mark Goddard
602f89baa6 Fix glance with file backend
Since https://review.opendev.org/647699/, we lost the logic to only
deploy glance-api on a single host when using the file backend.

This code was always a bit custom, and would be better supported by
using the 'host_in_groups' pattern we have in a few other places where a
single group name does not describe the placement of containers for a
service.

Change-Id: I21ce4a3b0beee0009ac69fecd0ce24efebaf158d
Closes-Bug: #1836151
2019-07-11 09:25:46 +01:00
Radosław Piliszek
b166d2550e Do not require valid migration_interface for controllers
Controllers lacking compute should not be required to provide
valid migration_interface as it is not used there (and prechecks
do not check that either).

Inclusion of libvirt conf section is now conditional on service type.
libvirt conf section has been moved to separate included file to
avoid evaluation of the undefined variable (conditional block did not
prevent it and using 'default' filter may hide future issues).
See https://github.com/ansible/ansible/issues/58835
Additionally this fixes the improper nesting of 'if' blocks for libvirt.

Change-Id: I77af534fbe824cfbe95782ab97838b358c17b928
Closes-Bug: #1835713
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-07-10 21:04:14 +02:00
Radosław Piliszek
0c00915c41 Enhance merge_* action plugins to allow expected relative includes
This mimics behavior of core 'template' module to allow relative
includes from the same dir as merged template, base dir of
playbook/role (usually role for us) and its 'templates' subdir.

Additionally old unused code was removed.

Change-Id: I83804d3cf5f17eb2302a2dfe49229c6277b1e25f
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-07-10 20:52:53 +02:00
Michal Nasiadka
4e3054b5da Add 'allow *' to getting ceph mds keyring
* Sometimes getting/creating ceph mds keyring fails, similar to https://tracker.ceph.com/issues/16255

Change-Id: I47587cbeb8be0e782c13ba7f40367409e2daa8a8
2019-07-10 13:09:38 +02:00
Zuul
8ec3ffc64b Merge "Fix nova deploy with Ansible<2.8" 2019-07-09 09:33:28 +00:00