353 Commits

Author SHA1 Message Date
Michal Nasiadka
4e6fe7a6da Remove kolla-ceph
Kolla-Ansible Ceph deployment mechanism has been deprecated in Train [1].

This change removes the Ansible code and associated CI jobs.

[1]: https://review.opendev.org/669214

Change-Id: Ie2167f02ad2f525d3b0f553e2c047516acf55bc2
2020-02-11 11:42:06 +01:00
Mark Goddard
3b24e566f6 CentOS 8: Deploy CentOS 8 containers
* HAProxy is now 1.8 in CentOS 8
* Support python3 in baremetal role
* Remove support for environments without python2 installed (this could
  not have worked since we gather facts before this point)

Workarounds:

* Using CentOS 7 yum repo for Docker, with module_hotfixes

Change-Id: I30bd3d58f6224ad4c9575ba66c74deabe6895cc4
Partially-Implements: blueprint centos-rhel-8
2020-01-28 16:46:41 +00:00
Zuul
5126087af5 Merge "CentOS 8: Support variable image tag suffix" 2020-01-21 09:29:58 +00:00
Marcin Juszkiewicz
934fea1aac Ansible lint: disable some checks
204 for very long url which is hard to break safely
306 for "echo | docker" as echo should not fall

Change-Id: I14df39d611d39ad1f6184ab92d628cb010881fbb
2020-01-17 13:51:43 +00:00
Mark Goddard
a6cb008c54 Ansible lint: task names
Change-Id: Iecbc2fe5fa3391dca5a3cc7e575314b95942114b
Co-Authored-By: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
2020-01-13 10:38:12 +00:00
Mark Goddard
9e3c774371 Ansible lint: remove comparison with true or false
Change-Id: I3caa4581ba276082e859f18aaa6638472f5fbe49
Co-Authored-By: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
2020-01-13 10:33:43 +00:00
Mark Goddard
9755c924be CentOS 8: Support variable image tag suffix
For the CentOS 7 to 8 transition, we will have a period where both
CentOS 7 and 8 images are available. We differentiate these images via a
tag - the CentOS 8 images will have a tag of train-centos8 (or
master-centos8 temporarily).

To achieve this, and maintain backwards compatibility for the
openstack_release variable, we introduce a new 'openstack_tag' variable.
This variable is based on openstack_release, but has a suffix of
'openstack_tag_suffix', which is empty except on CentOS 8 where it has a
value of '-centos8'.

Change-Id: I12ce4661afb3c255136cdc1aabe7cbd25560d625
Partially-Implements: blueprint centos-rhel-8
2020-01-10 09:56:04 +00:00
Pierre Riteau
49f334284b Template custom HAProxy service configuration
Change Id84e3b6e62e544582d6917047534e846e026798d added support for
custom HAProxy service config using a plain copy of files in services.d.

Use a template action instead of a copy so that we can use variables and
iterate over group of hosts.

Change-Id: I1f07785932de4e4540422bd18af95241f05a67bf
2019-12-10 15:31:45 +01:00
Zuul
cfdcaf3ab2 Merge "Fixes the compatibility issue at HAProxy role with Ansible 2.9" 2019-11-23 17:02:25 +00:00
Michal Nasiadka
1009931162 Change local_action to delegate_to: localhost
As part of the effort to implement Ansible code linting in CI
(using ansible-lint) - we need to implement recommendations from
ansible-lint output [1].

One of them is to stop using local_action in favor of delegate_to -
to increase readability and and match the style of typical ansible
tasks.

[1]: https://review.opendev.org/694779/

Partially implements: blueprint ansible-lint

Change-Id: I46c259ddad5a6aaf9c7301e6c44cd8a1d5c457d3
2019-11-22 15:04:44 +00:00
Dincer Celik
4f216b3d28 Fixes the compatibility issue at HAProxy role with Ansible 2.9
Change-Id: I36b858b9b03005feabd77e7208674f37c820e9d4
Closes-Bug: #1852430
2019-11-14 12:56:28 +03:00
Keith Plant
a3a1b068fe Add support for custom HAProxy service config
Allow users to create/override HAProxy service configuration by
copying over '*.cfg' files from {{ node_custom_config
}}/haproxy/services.d/

Ex: /etc/kolla/config/haproxy/services.d/radosgw.cfg

Change-Id: Id84e3b6e62e544582d6917047534e846e026798d
Signed-off-by: Keith Plant <kplantjr@gmail.com>
2019-11-11 15:24:46 -05:00
Radosław Piliszek
bc053c09c1 Implement IPv6 support in the control plane
Introduce kolla_address filter.
Introduce put_address_in_context filter.

Add AF config to vars.

Address contexts:
- raw (default): <ADDR>
- memcache: inet6:[<ADDR>]
- url: [<ADDR>]

Other changes:

globals.yml - mention just IP in comment

prechecks/port_checks (api_intf) - kolla_address handles validation

3x interface conditional (swift configs: replication/storage)

2x interface variable definition with hostname
(haproxy listens; api intf)

1x interface variable definition with hostname with bifrost exclusion
(baremetal pre-install /etc/hosts; api intf)

neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network

basic multinode source CI job for IPv6

prechecks for rabbitmq and qdrouterd use proper NSS database now

MariaDB Galera Cluster WSREP SST mariabackup workaround
(socat and IPv6)

Ceph naming workaround in CI
TODO: probably needs documenting

RabbitMQ IPv6-only proto_dist

Ceph ms switch to IPv6 mode

Remove neutron-server ml2_type_vxlan/vxlan_group setting
as it is not used (let's avoid any confusion)
and could break setups without proper multicast routing
if it started working (also IPv4-only)

haproxy upgrade checks for slaves based on ipv6 addresses

TODO:

ovs-dpdk grabs ipv4 network address (w/ prefix len / submask)
not supported, invalid by default because neutron_external has no address
No idea whether ovs-dpdk works at all atm.

ml2 for xenapi
Xen is not supported too well.
This would require working with XenAPI facts.

rp_filter setting
This would require meddling with ip6tables (there is no sysctl param).
By default nothing is dropped.
Unlikely we really need it.

ironic dnsmasq is configured IPv4-only
dnsmasq needs DHCPv6 options and testing in vivo.

KNOWN ISSUES (beyond us):

One cannot use IPv6 address to reference the image for docker like we
currently do, see: https://github.com/moby/moby/issues/39033
(docker_registry; docker API 400 - invalid reference format)
workaround: use hostname/FQDN

RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4.
This is due to old RabbitMQ versions available in images.
IPv4 is preferred by default and may fail in the IPv6-only scenario.
This should be no problem in real life as IPv6-only is indeed IPv6-only.
Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will
no longer be relevant as we supply all the necessary config.
See: https://github.com/rabbitmq/rabbitmq-server/pull/1982

For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed
to work well). Older Ansible versions are known to miss IPv6 addresses
in interface facts. This may affect redeploys, reconfigures and
upgrades which run after VIP address is assigned.
See: https://github.com/ansible/ansible/issues/63227

Bifrost Train does not support IPv6 deployments.
See: https://storyboard.openstack.org/#!/story/2006689

Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c
Implements: blueprint ipv6-control-plane
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-10-16 10:24:35 +02:00
Viktor Michalek
78aab26295 Haproxy: fix haproxy_cmd for Debian
Change-Id: I097082112b857444c3e2f73896be5832a776743b
2019-10-03 04:33:22 -07:00
Kris Lindgren
2fe0d98ebb Add a job that *only* deploys updated containers
Sometimes as cloud admins, we want to only update code that is running
in a cloud.  But we dont need to do anything else.  Make an action in
kolla-ansible that allows us to do that.

Change-Id: I904f595c69f7276e71692696471e32fd1f88e6e8
Implements: blueprint deploy-containers-action
2019-09-26 17:51:14 +01:00
Zuul
1982c9809f Merge "Add <project>_install_type for all projects" 2019-09-24 16:05:38 +00:00
Mark Goddard
6f05f1b844 Ensure keepalived is restarted during upgrade
During upgrade, we stop all slave keepalived containers. However, if the
keepalived container configuration has not changed, we never restart
them.

This change fixes the issue by notifying the restart handler when the
containers are stopped.

Change-Id: Ibe094b0c14a70a0eb811182d96f045027aa02c2a
Closes-Bug: #1836368
2019-09-23 15:27:34 +01:00
Mark Goddard
cc555c4196 Add <project>_install_type for all projects
This allows the install type for the project to be different than
kolla_install_type

This can be used to avoid hitting bug 1786238, since kuryr only supports
the source type.

Change-Id: I2b6fc85bac092b1614bccfd22bee48442c55dda4
Closes-Bug: #1786238
2019-09-23 10:23:54 +00:00
Radosław Piliszek
70b4bf6cbf Fix for haproxy precheck failing on CentOS running non-root
Change-Id: I7f2b3a6f1eacd4cabcaa31de543b7489bc5e654b
Closes-bug: #1844636
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-09-19 11:02:28 +02:00
Radosław Piliszek
b4ef4638a6 Fix enforced horizon redirect to https
Also fixes similar issues introduced by the same recent change.
Added FIXME note about possible TLS malfunction regarding horizon.

Change-Id: I5f46a9306139eb550d3849757c8bdf0767537c78
Closes-Bug: #1844016
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-09-14 22:00:56 +02:00
Zuul
ff86c2f2e3 Merge "Implement TLS encryption for internal endpoints" 2019-09-12 09:20:54 +00:00
Zuul
f960a5b58e Merge "HAProxy backend connection limits" 2019-08-27 12:58:07 +00:00
Krzysztof Klimonda
b0ecd8b67c Implement TLS encryption for internal endpoints
This review is the first one in a series of patches and it introduces an
optional encryption for internal openstack endpoints, implementing part
of the add-ssl-internal-network spec.

Change-Id: I6589751626486279bf24725f22e71da8cd7f0a43
2019-08-22 16:39:21 -07:00
Kien Nguyen
577bb50a04 Add Masakari Ansible role
Masakari provides Instances High Availability Service for
OpenStack clouds by automatically recovering failed Instances.

Depends-On: https://review.openstack.org/#/c/615469/
Change-Id: I0b3457232ee86576022cff64eb2e227ff9bbf0aa
Implements: blueprint ansible-masakari
Co-Authored-By: Gaëtan Trellu <gaetan.trellu@incloudus.com>
2019-08-15 09:58:53 -04:00
Scott Solkhon
46f9ad3a96 HAProxy backend connection limits
The default connection limits for backends is 2000
however, mariadb defaults to a max of 10000 conections,
therefore changing this limit to match the mariadb limit.

'haproxy_max_connections' also needs to be bumped
for this to work.

Change-Id: I5ded328485855f3f3d4390282040b0d89d08d997
2019-08-14 10:44:31 +00:00
Mark Goddard
b123bf6621 Use become for all docker tasks
Many tasks that use Docker have become specified already, but
not all. This change ensures all tasks that use the following
modules have become:

* kolla_docker
* kolla_ceph_keyring
* kolla_toolbox
* kolla_container_facts

It also adds become for 'command' tasks that use docker CLI.

Change-Id: I4a5ebcedaccb9261dbc958ec67e8077d7980e496
2019-06-06 19:04:58 +01:00
Zuul
2208b0214e Merge "Adds Qinling Ansible role" 2019-06-03 20:29:41 +00:00
Gaetan Trellu
edb3489820 Adds Qinling Ansible role
Qinling is an OpenStack project to provide "Function as a Service".
This project aims to provide a platform to support serverless functions.

Change-Id: I239a0130f8c8b061b531dab530d65172b0914d7c
Implements: blueprint ansible-qinling-support
Story: 2005760
Task: 33468
2019-05-31 10:25:28 -04:00
binhong.hua
12ff28a693 Make kolla-ansible support extra volumes
When integrating 3rd party component into openstack with kolla-ansible,
maybe have to mount some extra volumes to container.

Change-Id: I69108209320edad4c4ffa37dabadff62d7340939
Implements: blueprint support-extra-volumes
2019-05-17 11:55:04 +08:00
Raimund Hook
6804a5a682 Ansible flush_handlers ignores conditional clauses
The flush_handlers clause doesn't honour conditional clauses.
Instead, it prints a warning and runs anyway:
[WARNING]: flush_handlers task does not support when conditional

See: https://github.com/ansible/ansible/pull/41126

TrivialFix

Change-Id: Iaf70c2e932ae6dfb723bdb2ba658acdbfe74ebe2
2019-05-09 11:51:59 +01:00
Raimund Hook
84ea42bd7c Updating Jinja filters to conform to Ansible 2.5+
Since Ansible 2.5, the use of jinja tests as filters has been
deprecated.

I've run the script provided by the ansible team to 'fix' the
jinja filters to conform to the newer syntax.

This fixes the deprecation warnings.

Change-Id: I844ecb7bec94e561afb09580f58b1bf83a6d00bd
Closes-bug: #1827370
2019-05-02 14:58:09 +01:00
Bai Yongjun
ed2fd243d1 Add cyborg to kolla-ansible
Because kolla-ansible not have cyborg so should add it.

Implements: blueprint add-cyborg-to-kolla-ansible

Depend-On: I497e67e3a754fccfd2ef5a82f13ccfaf890a6fcd

Change-Id: I6f7ae86f855c5c64697607356d0ff3161f91b239
2019-03-08 10:46:53 +08:00
Maciej Kucia
0d32b76a33 haproxy: Support for external IPv6 (VIP)
This change allows usage of IPv6 as public address

Change-Id: Ie82ec5fb0ac9106b39948c67d34d5ef611a8fa21
Signed-off-by: Maciej Kucia <m.kucia@partner.samsung.com>
2019-02-07 15:56:35 +01:00
Zuul
3af135d4fb Merge "Allow disabling keepalived for external LBs" 2018-12-17 12:52:07 +00:00
Zuul
63eccc6372 Merge "Remove the deprecate the Glance Registry" 2018-12-01 07:08:36 +00:00
Eduardo Gonzalez
1a682fab28 Support stop specific containers
With this change, an operator may be able to stop a
service container without stopping all services in a host.
This change is the starting point to start
fast-forward upgrades support.
In next changes new flags will be introducced to disable
stop dataplane services during upgrades.

Change-Id: Ifde7a39d7d8596ef0d7405ecf1ac1d49a459d9ef
Implements: blueprint support-stop-containers
2018-11-26 08:07:01 +00:00
caoyuan
03fd9715c5 Remove the deprecate the Glance Registry
A spec to Deprecate the Glance Registry Service[0] was accepted in Newton,
but it contained the ambiguous statement, "Mark the service as deprecated
and ready for removal in the Q release." kolla-ansible disable the
glance-registry in Q release[1], and since we are in S now,
remove glance-registry is safe.

[0]: http://specs.openstack.org/openstack/glance-specs/specs/newton/approved/glance/deprecate-registry.html
[1]: https://review.openstack.org/#/c/566804/

Change-Id: I48f794029e97aa6f76bbd500e33f28f51a3f2ac4
2018-11-21 20:51:51 +08:00
Doug Szumski
712c89760c Add support for deploying Monasca Grafana
The Monasca Grafana fork allows users to log into Grafana with their
OpenStack user credentials and see metrics associated with their
OpenStack project. The long term goal is to enable Keystone support
in upstream Grafana, but this work seems to have stalled.

Partially-Implements: blueprint monasca-grafana
Change-Id: Icc04613b2571c094ae23b66d0bcc38b58c0ee4e1
2018-11-02 13:35:35 +00:00
Doug Szumski
195fec4d8d Add missing HAProxy port checks for Monasca
TrivialFix
Change-Id: Iaf216016a6acf0e9c87fdb6b8902416f4849efa3
2018-11-02 13:04:06 +00:00
Cédric Jeanneret
778dba94a4 Load known, standard kernel modules from the host, not within containers
Known kernel modules are:
- dm-multipath (for multipathd)
- ip_vs (for keepalived)
- iscsi_tcp (for ironic-conductor)
- openvswitch (for openvswitch-vswitchd)

Change-Id: I1841ec30cde142c8019830ad3190847dfe493eb9
2018-10-11 10:26:34 +02:00
Adam Harwell
f1c8136556 Refactor haproxy config (split by service) V2.0
Having all services in one giant haproxy file makes altering
configuration for a service both painful and dangerous. Each service
should be configured with a simple set of variables and rendered with a
single unified template.

Available are two new templates:

* haproxy_single_service_listen.cfg.j2: close to the original style, but
only one service per file
* haproxy_single_service_split.cfg.j2: using the newer haproxy syntax
for separated frontend and backend

For now the default will be the single listen block, for ease of
transition.

Change-Id: I6e237438fbc0aa3c89a3c8bd706a53b74e71904b
2018-09-26 03:30:38 -07:00
Clint Byrum
a05e77d71a Allow disabling keepalived for external LBs
In some cases a deployer may want to use haproxy for SSL termination but
has external infrastructure for load balancing, and thus no need for
keepalived to manage the VIP.

Co-Authored-By: Adam Harwell <flux.adam@gmail.com>

Change-Id: I451d7e33f1e631038a8d198dbc33c9a8850571b7
2018-09-17 22:04:14 +00:00
jiangpch
d57c7019a9 Make haproxy proxy to the right glance_api backend
Since glance_api only start one container when using file
backend, the haproxy should follow this rule.

See: https://review.openstack.org/#/c/448654

Closes-Bug: #1722422

Change-Id: Id3519581e0f54509dacd24d0dd542c630342c771
2018-09-13 15:24:02 +01:00
Zuul
acb72ddbe1 Merge "Fix prechecks for adding a new haproxy node" 2018-08-15 09:56:20 +00:00
Zuul
3e45b2cbec Merge "Use include_tasks instead of include" 2018-07-27 08:16:08 +00:00
Zuul
d1e427b7c1 Merge "Apply Resource Constraints to Openstack Services" 2018-07-26 14:18:34 +00:00
Lakshmi Prasanna Goutham Pratapa
76210a2d85 Apply Resource Constraints to Openstack Services
This commit is to apply resource-constraints only to few OpenStack services.
Commit to apply constraints to other services will be made in coming commits.

Partially-Implements: blueprint resource-constraints

Change-Id: Icafa54baca24d2de64238222a5677b9d8b90e2aa
2018-07-25 17:05:04 +01:00
Jeffrey Zhang
b51eeed89e Use include_tasks instead of include
include is marked as deprecated since ansible 2.4[0]

[0] https://docs.ansible.com/ansible/2.4/include_module.html#deprecated

Co-Authored-By: confi-surya <singh.surya64mnnit@gmail.com>
Change-Id: Ic9d71e1865d1c728890625aeddf424a5734c0a8a
2018-07-25 23:57:22 +08:00
Victor Coutellier
6a9a0e927d Fix prechecks for adding a new haproxy node
Change-Id: I2615e4938ec6b4a525f7fddde5a51a139dced8de
Closes-Bug: #1783381
2018-07-24 14:00:32 -04:00
Kevin Tibi
16df54eaa5 Disable TLS 1.1 on haproxy
While it is possible to implement countermeasures against some attacks
on TLS, migrating to a later version of TLS (TLS 1.2 is strongly
encouraged) is the only reliable method to protect against
the current protocol vulnerabilities.[1]

[1] https://blog.pcisecuritystandards.org/are-you-ready-for-30-june-2018-sayin-goodbye-to-ssl-early-tls

Change-Id: I44f67e3a49bb00fea069d29c46b3e86404c7df0b
2018-07-20 11:10:33 +02:00