Resolves a number of TODOs in the CI configuration that provide support
for upgrading from the Stein release.
Change-Id: I9bac5c230b82ac7c097fe6ca2556e428abda31a1
Depends-On: https://review.opendev.org/694254
Tests the following operations for MariaDB:
* Stop
* Recovery
Backup and restore will be added in a separate change.
Depends-On: https://review.opendev.org/693329
Change-Id: I836d91554715cce0e82c1bbebb7430c457418b2d
Commit 73b6a66fd4db4345e5c1ed8acf2f3d10170bfdd4 added installation of
Python 3 package. But without root permissions it fails.
Change-Id: I65ca794955a1b1419853bf63be36cb0d1f2d2345
Introduce kolla_address filter.
Introduce put_address_in_context filter.
Add AF config to vars.
Address contexts:
- raw (default): <ADDR>
- memcache: inet6:[<ADDR>]
- url: [<ADDR>]
Other changes:
globals.yml - mention just IP in comment
prechecks/port_checks (api_intf) - kolla_address handles validation
3x interface conditional (swift configs: replication/storage)
2x interface variable definition with hostname
(haproxy listens; api intf)
1x interface variable definition with hostname with bifrost exclusion
(baremetal pre-install /etc/hosts; api intf)
neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network
basic multinode source CI job for IPv6
prechecks for rabbitmq and qdrouterd use proper NSS database now
MariaDB Galera Cluster WSREP SST mariabackup workaround
(socat and IPv6)
Ceph naming workaround in CI
TODO: probably needs documenting
RabbitMQ IPv6-only proto_dist
Ceph ms switch to IPv6 mode
Remove neutron-server ml2_type_vxlan/vxlan_group setting
as it is not used (let's avoid any confusion)
and could break setups without proper multicast routing
if it started working (also IPv4-only)
haproxy upgrade checks for slaves based on ipv6 addresses
TODO:
ovs-dpdk grabs ipv4 network address (w/ prefix len / submask)
not supported, invalid by default because neutron_external has no address
No idea whether ovs-dpdk works at all atm.
ml2 for xenapi
Xen is not supported too well.
This would require working with XenAPI facts.
rp_filter setting
This would require meddling with ip6tables (there is no sysctl param).
By default nothing is dropped.
Unlikely we really need it.
ironic dnsmasq is configured IPv4-only
dnsmasq needs DHCPv6 options and testing in vivo.
KNOWN ISSUES (beyond us):
One cannot use IPv6 address to reference the image for docker like we
currently do, see: https://github.com/moby/moby/issues/39033
(docker_registry; docker API 400 - invalid reference format)
workaround: use hostname/FQDN
RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4.
This is due to old RabbitMQ versions available in images.
IPv4 is preferred by default and may fail in the IPv6-only scenario.
This should be no problem in real life as IPv6-only is indeed IPv6-only.
Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will
no longer be relevant as we supply all the necessary config.
See: https://github.com/rabbitmq/rabbitmq-server/pull/1982
For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed
to work well). Older Ansible versions are known to miss IPv6 addresses
in interface facts. This may affect redeploys, reconfigures and
upgrades which run after VIP address is assigned.
See: https://github.com/ansible/ansible/issues/63227
Bifrost Train does not support IPv6 deployments.
See: https://storyboard.openstack.org/#!/story/2006689
Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c
Implements: blueprint ipv6-control-plane
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
This ensures that failure of a single host fails the whole play at that
task. This can avoid confusing errors such as when the task
"Assert that the nodepool private IPv4 address is assigned" fails on one
host, causing subsequent errors on other hosts.
Note that this only affects the Zuul playbooks, not Kolla Ansible's
playbooks.
Change-Id: I77a6534dd2ddd188f795e17d17a44be249d01f31
After modernising docker configuration
(I1215e04ec15b01c0b43bac8c0e81293f6724f278), we lost our
registry-mirrors configuration in CI that lets us use a mirror of
Dockerhub.
This change uses the new docker_custom_config variable to configure the
registry mirror.
Change-Id: I1430413c12e9d0b59e4f216ff66372de0f3a4f21
VXLAN is necessary to run HA in CI (due to floating VIP
address handled by keepalived).
It also turned out to be required to have private
IPv6 address assignments.
This patch is based on linux bridge rather than OVS
to avoid problems with OVS deployed in containers.
This patch enables haproxy in multinode jobs.
Includes saving of linux networking details.
Makes DASHBOARD_URL agree with OS_AUTH_URL - properly uses the
pre-upgrade value for testing.
Co-authored-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Depends-on: https://review.opendev.org/683068
Depends-on: https://review.opendev.org/682957
Change-Id: I66888712da80c3d6f84ee4949762961664d3adea
This lets us control the upgrade process entirely from the
current branch.
Change-Id: Ic8c39e415846596c23dae93c2839375a24e8b888
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
These filters can be used to capture a lot of the logic that we
currently have in 'when' statements, about which services are enabled
for a particular host.
In order to use these filters, it is necessary to install the
kolla_ansible python module, and not just the dependencies listed in
requirements.txt. The CI test and quickstart install from source
documentation has been updated accordingly.
Ansible is not currently in OpenStack global requirements, so for unit
tests we avoid a direct dependency on Ansible and provide fakes where
necessary.
Change-Id: Ib91cac3c28e2b5a834c9746b1d2236a309529556
- Test Zun on CentOS too
- Make etcd change also trigger Zun jobs (like kuryr and zun)
- Test multinode Zun deployments instead of AIO
(more likely to break)
- In Zun scenario, stop configuring docker for legacy swarm mode
(Zun is no swarm)
- Separate test-zun.sh testing script
- Show appcontainer to see which node it has been started on
Change-Id: I289b1009fe00aedb9b78cbd83298b14da5fd9670
Depends-On: https://review.opendev.org/676736
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Multinode jobs did not run sanity checks for all the hosts,
only primary. Now they check all.
Additionally upgrades are now checked using the proper
(pre-upgrade) scripts (not that it matters too much as they
are the same atm) and both checks are done, not only failures,
but also config.
Change-Id: I10552e256edbddd5b1f8a8a7f8805262e72ce8d8
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Some kolla-ansible jobs failed due to using external mirrors
instead of local ones.
This was due to not using the template override provided by kolla.
This patch fixes that.
Depends-On: https://review.opendev.org/668226
Change-Id: I27f714fdf05e521aa8ce25c5683a452ceb35eeb8
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Adds four new CI jobs for testing centos/ubuntu binary/source deploys
with ironic enabled. These are run only when there are changes to the
ironic role.
Performs some simple testing by creating a node using the fake-hardware
hardware type and creating a server.
Change-Id: Ie669e57ce2af53257b4ca05f45193cb73f48827a
Depends-On: https://review.opendev.org/664011
Add CI jobs for testing an upgrade of a multinode system with Ceph
enabled. As for the existing upgrade job, we upgrade from the previous
release to the current release.
Change-Id: I931772ca4c63757769467a57c80dc0726a11167a
Depends-On: https://review.opendev.org/658163
Periodic jobs don't have zuul.change defined, since there is no change
being tested. This causes an early failure when referencing zuul.change
to set the image tag for built images. In periodic jobs we'll never need
to build images because there is no dependent kolla change under test.
Change-Id: I6d9d81cf17b7d0d7aaf87cd96418c904c46681f2
Typically, non-executable files should have 660 or 600 and executable
files and directories should have 770. All should be owned by the
'config_owner_user' and 'config_owner_group' variables.
This change adds a script to check the owner and permissions of config
files under /etc/kolla, and runs it at the end of CI jobs.
Change-Id: Icdbabf36e284b9030017a0dc07b9dc81a37758ab
Related-Bug: #1821579
This patch adds two new jobs:
* kolla-ansible-centos-source-upgrade
* kolla-ansible-ubuntu-source-upgrade
These jobs first deploy a control plane using the previous release of
Kolla Ansible, then upgrade to the current release.
Because we can't change the branch of the git repository on the Zuul
executor, we change the branch of the kolla-ansible repository on the
primary node to the branch of the previous release, in this case
stable/rocky. A new remote-template role has been added that supports
generating templates using a remote template source, to generate config
files using the previous kolla-ansible branch.
If the change being tested depends on a kolla change for the current
branch, then we build images. Rather than using the current
kolla-ansible version to tag the images, we now tag them with
change_<gerrit change ID>. This is because the version of kolla-ansible
will change from the previous release to the current one as we upgrade
the system.
Finally, it should be noted that the 'previous_release' variable in the
Zuul config needs to be updated with each release, since this sets the
release of kolla-ansible that is installed initially.
Depends-On: https://review.openstack.org/645089/
Depends-On: https://review.openstack.org/644250/
Depends-On: https://review.openstack.org/645816/
Depends-On: https://review.openstack.org/645840/
Change-Id: If301e0affcd55360fefe3b105f023ae5c47b0853
Since Rocky bluestore is the default store type used by Kolla Ceph.
Filestore is still used by the customers. The patch allows deploy
both Ceph bluestore OSD and Ceph filestore OSD in Kolla-ansible CI
jobs.
Change-Id: Iaa7c1b2ff35eb649c3a6d9f7266463d8961cdc9e
Signed-off-by: tone.zhang <tone.zhang@arm.com>
Kolla Ansible's bootstrap-servers command provides support for
installing the Docker engine. This is currently done using the packages
at https://apt.dockerproject.org and https://yum.dockerproject.org.
These packages are outdated, with the most recent packages from May 2017
- docker-engine-17.05.
The source for up to date docker packages is
https://download.docker.com, which was introduced with the move to
Docker Community Edition (CE) and Docker Enterprise Edition (EE).
This change adds support to bootstrap-servers for Docker CE for CentOS
and Ubuntu.
It also adds a new variable, 'enable_docker_repo', which controls
whether a package repository for Docker will be enabled.
It also adds a new variable, 'docker_legacy_packages', which controls
whether the legacy packages at dockerproject.org will be used or the
newer packages at docker.com. The default value for this variable is
'false', meaning to use Docker CE.
Upgrading from docker-engine to docker-ce has been tested on CentOS 7.5
and Ubuntu 16.04, by running 'kolla-ansible bootstrap-servers' with
'docker_legacy_packages' set to 'false'. The upgrades were successful,
but result in all containers being stopped. For this reason, the
bootstrap-servers command checks running containers prior to upgrading
packages, and ensures they are running after the package upgrade is
complete.
As mentioned in the release note, care should be taken when upgrading
Docker with clustered services, which could lose quorum. To avoid this,
use --serial or --limit to apply the change in batches.
Change-Id: I6dfd375c868870f8646ef1a8f02c70812e8f6271
Implements: blueprint docker-ce
This change is intended to ease the addition of new test scenarios, by
splitting the existing setup_gate.sh script up into several smaller
scripts. We may wish to further break these scripts up in future,
depending on the requirements of new test jobs.
Change-Id: Ifa442ac79a54f749c56d20f5f7383c3852d95268
Kolla Ansible now claims [1] to support executing as a user other than
root. We should ensure that this is tested in CI.
This change removes the 'become' from hosts in the inventory, and sets
the remote user to 'kolla', as configured via the bootstrap-servers
command. The bootstrap-servers command and other ansible commands
executed before it still need to execute as the zuul user and not as
kolla, since kolla does not exist yet.
The autogenerated SSH private key in passwords.yml is now added to the
zuul user's SSH config, such that it can SSH as the kolla user, which
has authorised this key.
[1]
https://blueprints.launchpad.net/kolla-ansible/+spec/ansible-specific-task-become
Change-Id: I8f3587e2908bc5b8889cd6fbc01981a6186d33e6
Deploys a bifrost container using kolla-ansible bifrost-deploy.
IPA and disk images are downloaded rather than built to improve
reliability.
Currently only minimal testing of the deployment is performed, creating
and deleting an ironic node. Ideally we would perform a bare metal node
deployment.
The job is based on CentOS, as Ubuntu bifrost deployment is currently
failing with a python-MySQLdb error.
Change-Id: Ic45094594c21116b5b0d6a606f568fc7954175e3
- Ceph images are not being built when using depends-on a kolla build
job.
- Sync inventory files with current ones which causes ceph to fail due
missing groups.
- Small corrections in ceph config.yml syntax.
- fix preparse the disk only once
- enable ceph nfs only when enable_ceph_nfs is true
Co-Authored-By: Jeffrey Zhang <zhang.lei.fly@gmail.com>
Change-Id: Id0c7963bf59e2af4944834dcd16589a638e78ba5
By default, the ansible task will gather facts, it can be configured by
set "father_facts: true", it's unncessary to run setup in tasks, it will
waste much time when hosts is large.
Change-Id: I9462a538c76b6271a34531d8752b72cc695f693e
Closes-Bug: #1743485
- move check container failure from post.yml to run.yml
- add binary related jobs
- use static kolla-ansible src dir, which is helpful for kolla project
to use.
- generate correct /etc/hosts by using private ip address and hostname
- fix the wrong api interface in global.yml file
Change-Id: Idfdee6dfe18f0fa2d4f984df59b57553122ce298
