27 Commits

Author SHA1 Message Date
Radosław Piliszek
a2fc684164 CI: Refactor a lot
Separate upgrade logic to is_upgrade job var and rename
scenarios to match.

Rename "ACTION" to "SCENARIO" (as it is a scenario).

Separate testing of dashboard (aka Horizon) and increase
its timeout to 5 minutes (CentOS 7 slow as always).

Separate initialization of core OpenStack.

Use gate setup script from ./tests/

Remove useless tox setupenv.

Do not deploy Heat when not really necessary.

Change-Id: I4fca319ccc3de7188f8b7b44c9c71321e3899467
2019-11-26 17:30:14 +01:00
Zuul
2fb08fa4e1 Merge "CI: Add mariadb test" 2019-11-15 12:17:13 +00:00
Mark Goddard
ed996ef90d CI: Add mariadb test
Tests the following operations for MariaDB:

* Stop
* Recovery

Backup and restore will be added in a separate change.

Depends-On: https://review.opendev.org/693329
Change-Id: I836d91554715cce0e82c1bbebb7430c457418b2d
2019-11-07 18:03:07 +00:00
Mark Goddard
e18e40eb6a CI: Don't always build images for Debian
Debian source images are published now.

Change-Id: Id611fd2fa71eb54ca08d1f68de9505d28ad4ea40
2019-11-07 10:32:45 +00:00
Zuul
f2042a878e Merge "Support multiple nova cells" 2019-10-17 08:44:23 +00:00
Zuul
e7a85726d2 Merge "Implement IPv6 support in the control plane" 2019-10-16 18:22:23 +00:00
Doug Szumski
78a828ef42 Support multiple nova cells
This patch adds initial support for deploying multiple Nova cells.

Splitting a nova-cell role out from the Nova role allows a more granular
approach to deploying and configuring Nova services.

A new enable_cells flag has been added that enables the support of
multiple cells via the introduction of a super conductor in addition to
cell-specific conductors. When this flag is not set (the default), nova
is configured in the same manner as before - with a single conductor.

The nova role now deploys the global services:

* nova-api
* nova-scheduler
* nova-super-conductor (if enable_cells is true)

The nova-cell role handles services specific to a cell:

* nova-compute
* nova-compute-ironic
* nova-conductor
* nova-libvirt
* nova-novncproxy
* nova-serialproxy
* nova-spicehtml5proxy
* nova-ssh

This patch does not support using a single cell controller for managing
more than one cell. Support for sharing a cell controller will be added
in a future patch.

This patch should be backwards compatible and is tested by existing CI
jobs. A new CI job has been added that tests a multi-cell environment.

ceph-mon has been removed from the play hosts list as it is not
necessary - delegate_to does not require the host to be in the play.

Documentation will be added in a separate patch.

Partially Implements: blueprint support-nova-cells
Co-Authored-By: Mark Goddard <mark@stackhpc.com>
Change-Id: I810aad7d49db3f5a7fd9a2f0f746fd912fe03917
2019-10-16 17:42:36 +00:00
Radosław Piliszek
f69a8a9bf4 CI: Increase timeout for upgrade jobs by 30 minutes
Upgrade jobs like to timeout in the 2-hour window when they must
build their images.
This increase is already applied in ceph jobs.

Change-Id: Ic1118760d9192cc15e1ebf37fb8adf3440f18a78
2019-10-16 10:55:44 +02:00
Radosław Piliszek
bc053c09c1 Implement IPv6 support in the control plane
Introduce kolla_address filter.
Introduce put_address_in_context filter.

Add AF config to vars.

Address contexts:
- raw (default): <ADDR>
- memcache: inet6:[<ADDR>]
- url: [<ADDR>]

Other changes:

globals.yml - mention just IP in comment

prechecks/port_checks (api_intf) - kolla_address handles validation

3x interface conditional (swift configs: replication/storage)

2x interface variable definition with hostname
(haproxy listens; api intf)

1x interface variable definition with hostname with bifrost exclusion
(baremetal pre-install /etc/hosts; api intf)

neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network

basic multinode source CI job for IPv6

prechecks for rabbitmq and qdrouterd use proper NSS database now

MariaDB Galera Cluster WSREP SST mariabackup workaround
(socat and IPv6)

Ceph naming workaround in CI
TODO: probably needs documenting

RabbitMQ IPv6-only proto_dist

Ceph ms switch to IPv6 mode

Remove neutron-server ml2_type_vxlan/vxlan_group setting
as it is not used (let's avoid any confusion)
and could break setups without proper multicast routing
if it started working (also IPv4-only)

haproxy upgrade checks for slaves based on ipv6 addresses

TODO:

ovs-dpdk grabs ipv4 network address (w/ prefix len / submask)
not supported, invalid by default because neutron_external has no address
No idea whether ovs-dpdk works at all atm.

ml2 for xenapi
Xen is not supported too well.
This would require working with XenAPI facts.

rp_filter setting
This would require meddling with ip6tables (there is no sysctl param).
By default nothing is dropped.
Unlikely we really need it.

ironic dnsmasq is configured IPv4-only
dnsmasq needs DHCPv6 options and testing in vivo.

KNOWN ISSUES (beyond us):

One cannot use IPv6 address to reference the image for docker like we
currently do, see: https://github.com/moby/moby/issues/39033
(docker_registry; docker API 400 - invalid reference format)
workaround: use hostname/FQDN

RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4.
This is due to old RabbitMQ versions available in images.
IPv4 is preferred by default and may fail in the IPv6-only scenario.
This should be no problem in real life as IPv6-only is indeed IPv6-only.
Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will
no longer be relevant as we supply all the necessary config.
See: https://github.com/rabbitmq/rabbitmq-server/pull/1982

For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed
to work well). Older Ansible versions are known to miss IPv6 addresses
in interface facts. This may affect redeploys, reconfigures and
upgrades which run after VIP address is assigned.
See: https://github.com/ansible/ansible/issues/63227

Bifrost Train does not support IPv6 deployments.
See: https://storyboard.openstack.org/#!/story/2006689

Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c
Implements: blueprint ipv6-control-plane
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-10-16 10:24:35 +02:00
Marcin Juszkiewicz
b9d6cc2a7f Add Debian/source CI job
Change-Id: I0628b16e3ebdb3fa8196acdc1bd9c63e75bcfb09
2019-10-04 08:57:55 +00:00
Radosław Piliszek
d4de1d7520 CI: Zun jobs
- Test Zun on CentOS too
- Make etcd change also trigger Zun jobs (like kuryr and zun)
- Test multinode Zun deployments instead of AIO
  (more likely to break)
- In Zun scenario, stop configuring docker for legacy swarm mode
  (Zun is no swarm)
- Separate test-zun.sh testing script
- Show appcontainer to see which node it has been started on

Change-Id: I289b1009fe00aedb9b78cbd83298b14da5fd9670
Depends-On: https://review.opendev.org/676736
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-08-16 17:11:00 +02:00
Kien Nguyen
fbac54c5f5 Testing Masakari role in gate
Add Masakari testing into the Gate.

Change-Id: I52df33f963e7d2ae4059887df3d24d9e6642134e
Depends-On: https://review.opendev.org/#/c/615469/
Depends-On: https://review.opendev.org/#/c/615715
Implements: blueprint ansible-masakari
Co-Authored-By: Gaëtan Trellu <gaetan.trellu@incloudus.com>
2019-08-14 12:32:51 -04:00
Zuul
54856a873f Merge "Disable and remove OracleLinux CI jobs" 2019-06-27 14:14:38 +00:00
Mark Goddard
845040ad3f Add CI job for ironic
Adds four new CI jobs for testing centos/ubuntu binary/source deploys
with ironic enabled. These are run only when there are changes to the
ironic role.

Performs some simple testing by creating a node using the fake-hardware
hardware type and creating a server.

Change-Id: Ie669e57ce2af53257b4ca05f45193cb73f48827a
Depends-On: https://review.opendev.org/664011
2019-06-11 10:22:04 +01:00
Mark Goddard
6b87514b14 Disable and remove OracleLinux CI jobs
We have agreed to remove support for Oracle Linux, starting by
disabling
the CI jobs.

http://lists.openstack.org/pipermail/openstack-discuss/2019-June/006896.html

Change-Id: I79f6ec98383b1f59b8317a3720f6f9923d6bb21d
2019-06-10 08:49:10 +01:00
Mark Goddard
78ee0287a7 Test Ceph upgrade in CI
Add CI jobs for testing an upgrade of a multinode system with Ceph
enabled. As for the existing upgrade job, we upgrade from the previous
release to the current release.

Change-Id: I931772ca4c63757769467a57c80dc0726a11167a
Depends-On: https://review.opendev.org/658163
2019-06-03 08:24:32 +00:00
Zuul
1b2503b5d4 Merge "Test upgrades in CI" 2019-03-28 12:31:08 +00:00
Mark Goddard
c23c9b2c25 Test upgrades in CI
This patch adds two new jobs:

* kolla-ansible-centos-source-upgrade
* kolla-ansible-ubuntu-source-upgrade

These jobs first deploy a control plane using the previous release of
Kolla Ansible, then upgrade to the current release.

Because we can't change the branch of the git repository on the Zuul
executor, we change the branch of the kolla-ansible repository on the
primary node to the branch of the previous release, in this case
stable/rocky. A new remote-template role has been added that supports
generating templates using a remote template source, to generate config
files using the previous kolla-ansible branch.

If the change being tested depends on a kolla change for the current
branch, then we build images. Rather than using the current
kolla-ansible version to tag the images, we now tag them with
change_<gerrit change ID>. This is because the version of kolla-ansible
will change from the previous release to the current one as we upgrade
the system.

Finally, it should be noted that the 'previous_release' variable in the
Zuul config needs to be updated with each release, since this sets the
release of kolla-ansible that is installed initially.

Depends-On: https://review.openstack.org/645089/
Depends-On: https://review.openstack.org/644250/
Depends-On: https://review.openstack.org/645816/
Depends-On: https://review.openstack.org/645840/
Change-Id: If301e0affcd55360fefe3b105f023ae5c47b0853
2019-03-27 16:13:49 +00:00
Michal Nasiadka
ab04ab931a Bump up timeout for ceph jobs
Currently ceph jobs are often getting TIMED_OUT, increasing limit.

Change-Id: I3c6684984930d55a56da846bd8c3f19df2754b06
2019-03-25 12:50:53 +01:00
Mark Goddard
39410b0198 Run jobs on bionic instead of xenial
Migrate to the latest Ubuntu LTS release 18.04 aka Bionic.  See [0] for
the big picture.

Also test running tox jobs on Bionic.

[0] https://etherpad.openstack.org/p/devstack-bionic

Change-Id: I96e7b8d17bc1e92716c04fdcf362c2adb08a2212
2019-03-18 14:46:56 +00:00
Mark Goddard
c41c9a0ce0 Add cinder-lvm CI job
Tests Cinder with an LVM/iSCSI backend, including attaching a volume to
an instance.

Change-Id: I6963b5e81d7ac88dd1bdfb94b62d9d63028b9786
2019-03-01 14:36:30 +00:00
Zuul
e9fda8664e Merge "Add Kolla Ceph filestore CI jobs" 2019-01-07 11:35:54 +00:00
tone.zhang
6786bc5b77 Add Kolla Ceph filestore CI jobs
Since Rocky bluestore is the default store type used by Kolla Ceph.
Filestore is still used by the customers. The patch allows deploy
both Ceph bluestore OSD and Ceph filestore OSD in Kolla-ansible CI
jobs.

Change-Id: Iaa7c1b2ff35eb649c3a6d9f7266463d8961cdc9e
Signed-off-by: tone.zhang <tone.zhang@arm.com>
2019-01-01 13:32:16 +08:00
Eduardo Gonzalez
535aba79a9 Add NFV orchestration services CI job
Change-Id: I33484d0961f1cd17ed05dd124eba68d99b4db83f
2018-12-21 11:43:22 +01:00
Hongbin LU
f57c1aec6c Add Zun scenario job for ubuntu
Introduce a job 'kolla-ansible-ubuntu-source-zun' to test kolla
with Zun enabled. To reduce CI resource, this job will be triggered
only if there are changes on the Zun's ansible roles.

Change-Id: I0ba207e1d3761da2d6992c5834d4f59e7e1d6628
2018-10-22 14:24:36 +00:00
Mark Goddard
9322db0ed2 Make source jobs voting and gating
As discussed in Kolla IRC meeting 26th July 2018 [1].

[1]
http://eavesdrop.openstack.org/meetings/kolla/2018/kolla.2018-07-25-15.00.log.html#l-77

Change-Id: I6fc46658a9516e44843f31a308504534a6b93d8d
2018-08-15 16:51:09 +01:00
Eduardo Gonzalez
9b05c94266 Split zuul.yaml into multiple files
With the current increase of zuul jobs and
service testing, is easier to maintain and
control what changes are done in different files.

Ideally jobs.yaml would be split into multiple files,
in example per distro, install type, aio or multinode,
etc

Change-Id: I78d3f1f00f96ef3042e94bba6b19e61d7fff4203
2018-08-07 23:06:52 +02:00