27 Commits

Author SHA1 Message Date
James Kirsch
c15dc20341 Configure services to use Certificate Authority
Include a reference to the globally configured Certificate Authority to
all services. Services use the CA to verify HTTPs connections.

Change-Id: I38da931cdd7ff46cce1994763b5c713652b096cc
Partially-Implements: blueprint support-trusted-ca-certificate-file
2020-01-13 11:00:11 -08:00
Radosław Piliszek
bc053c09c1 Implement IPv6 support in the control plane
Introduce kolla_address filter.
Introduce put_address_in_context filter.

Add AF config to vars.

Address contexts:
- raw (default): <ADDR>
- memcache: inet6:[<ADDR>]
- url: [<ADDR>]

Other changes:

globals.yml - mention just IP in comment

prechecks/port_checks (api_intf) - kolla_address handles validation

3x interface conditional (swift configs: replication/storage)

2x interface variable definition with hostname
(haproxy listens; api intf)

1x interface variable definition with hostname with bifrost exclusion
(baremetal pre-install /etc/hosts; api intf)

neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network

basic multinode source CI job for IPv6

prechecks for rabbitmq and qdrouterd use proper NSS database now

MariaDB Galera Cluster WSREP SST mariabackup workaround
(socat and IPv6)

Ceph naming workaround in CI
TODO: probably needs documenting

RabbitMQ IPv6-only proto_dist

Ceph ms switch to IPv6 mode

Remove neutron-server ml2_type_vxlan/vxlan_group setting
as it is not used (let's avoid any confusion)
and could break setups without proper multicast routing
if it started working (also IPv4-only)

haproxy upgrade checks for slaves based on ipv6 addresses

TODO:

ovs-dpdk grabs ipv4 network address (w/ prefix len / submask)
not supported, invalid by default because neutron_external has no address
No idea whether ovs-dpdk works at all atm.

ml2 for xenapi
Xen is not supported too well.
This would require working with XenAPI facts.

rp_filter setting
This would require meddling with ip6tables (there is no sysctl param).
By default nothing is dropped.
Unlikely we really need it.

ironic dnsmasq is configured IPv4-only
dnsmasq needs DHCPv6 options and testing in vivo.

KNOWN ISSUES (beyond us):

One cannot use IPv6 address to reference the image for docker like we
currently do, see: https://github.com/moby/moby/issues/39033
(docker_registry; docker API 400 - invalid reference format)
workaround: use hostname/FQDN

RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4.
This is due to old RabbitMQ versions available in images.
IPv4 is preferred by default and may fail in the IPv6-only scenario.
This should be no problem in real life as IPv6-only is indeed IPv6-only.
Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will
no longer be relevant as we supply all the necessary config.
See: https://github.com/rabbitmq/rabbitmq-server/pull/1982

For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed
to work well). Older Ansible versions are known to miss IPv6 addresses
in interface facts. This may affect redeploys, reconfigures and
upgrades which run after VIP address is assigned.
See: https://github.com/ansible/ansible/issues/63227

Bifrost Train does not support IPv6 deployments.
See: https://storyboard.openstack.org/#!/story/2006689

Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c
Implements: blueprint ipv6-control-plane
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-10-16 10:24:35 +02:00
Keith Plant
b95ff2d1db Configure Telegraf to monitor Docker containers
Added configuration to ansible/roles/telegraf/templates/telegraf.conf.j2 to
allow telegraf to grab telemetry data from docker directly.

Added option to etc/kolla/globals.yml to switch on/off the configuration to
ingest data from the docker daemon into telegraf.

Change-Id: Icbebc415d643a237fa128840d5f5a9c91d22c12d
Signed-off-by: Keith Plant <kplantjr@gmail.com>
2019-08-13 08:17:00 -04:00
pangliye
93e868360d Add Kafka input to telegraf config
Change-Id: I9a8d3dc5f311d4ea4e5d9b03d522632abc66a7ac
2019-08-05 07:26:46 +00:00
Nick Jones
683f65aaf3
Fix Telegraf configuration template
Remove the erroneous curly braces that prevent the Telegraf
configuration template from being rendered.

Change-Id: Ibb8738b258245b4e43d5d877b156ed296fc66e4e
Closes-Bug: 1810505
2019-01-04 12:11:39 +00:00
pangliye
22d3857ce7 add monitoring of zookeeper
add monitoring of zookeeper

Change-Id: I1db23b67dd8f83f9341a376d9cd2d9a49abd4427
2018-11-13 03:46:20 +00:00
pangliye
daf5a637a3 Add monitoring of redis
add monitoring of redis

Change-Id: Ic6becef0c528083d48458c8dcaa10a2ca6c238a9
2018-11-08 06:23:26 +00:00
Kevin TIBI
7764ed9894 Fix telegraf output for influxdb
Kolla add multiple url in one outputs.influxdb.
This configuration is when influxdb is in cluster mode.
We need to write one conf by influxdb server.

Change-Id: I1c788b3c8400e74d50cace5a334311408989c5e1
2018-03-06 09:36:12 +00:00
Alexandru Bogdan Pica
8e3b79440c Implement external MariaDB and pre-configured Databases support
This change allows the following use cases:

1. Using an already-configured MariaDB / MySQL server / Cluster
2. Using already-created DB users, without requiring root DB access.

Update: added external mariadb precheck

Change-Id: I78b0d178306d7c5293b0bf53e445f19f18b4b824
Implements: blueprint external-mariadb-support.
Closes-Bug: #1603121
2018-01-23 13:07:40 +00:00
Zuul
8ad8342bc1 Merge "Telegraf should only output to influxdb when influxdb is enabled" 2017-11-28 14:47:57 +00:00
Christian Berendt
6d710f595c Collect chrony metrics with telegraf
Change-Id: I4cf1716c94e4fd808b66169348b07fa42e5b242e
Depends-on: I8fbef57b99122b8a093150f6ca1efdd33e032917
2017-11-02 20:47:45 +01:00
Christian Berendt
72dbef13dd Telegraf should only output to influxdb when influxdb is enabled
Change-Id: Ib0512481aebf56b26f93955e682ea0cb80cb5bc4
2017-10-25 00:08:41 +02:00
guochao
d787b2bb21 Add collectd support to telegraf container
Telegraf can collect sample from collectd via UDP.
Configure collectd to exposed samples and telegraf to collect them.

Change-Id: Ie868a4c23cb793e6054e14ee1209a56256c130ac
Depends-On: I33b1a3fe606bf5bd4a08901c7bf274aed5303055
2017-07-21 03:35:30 +00:00
Jenkins
ae7fe6bdbb Merge "Modify the configuration file of telegraf" 2017-07-06 04:11:44 +00:00
Paul Bourke
d8fe3ea780 Add a new 'outward' rabbitmq instance
Certain services such as Murano and trove require access to a rabbitmq
instance from tenant networks. [0]

Exposing the internal rabbitmq to end users is a security hole, hence
there are two options, 1) use vhosts in the existing rabbitmq, or two a
separate rabbitmq instances. Given the importance of rabbitmq to the
OpenStack deployment, we have decided to go with a separate instance.
Refer to [1] for more detail on the various options.

This change makes the rabbitmq role generic so that it can be reused, in
this case to start 'outward_rabbitmq'. It needs to be exposed via
haproxy both for network isolation and also because this is what Murano
configuration requires.

Follow on patches will be added to add a vhost in this outward instance
for Murano and other services which require access.

Based on the original work by bdaca[2]

[0] http://murano.readthedocs.io/en/stable-liberty/intro/architecture.html
[1] http://lists.openstack.org/pipermail/openstack-dev/2016-December/109091.html
[2] https://review.openstack.org/#/c/374525

Change-Id: Ib2bcc7ed4bf4f883a7cd1dfad3db89201e3cfd8d
Partial-Bug: #1620374
Depends-On: I020eb6219f89a310451becde41f6f1c7f54baadd
Co-Authored-By: Bartłomiej Daca <bartek.daca@gmail.com>
2017-06-15 11:12:22 +00:00
Bertrand Lallau
afdd11b9a2 Generalize api_interface_address variable usage
Useful api_interface_address variable has been define here:
https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L57
In order to simplify codebase we must use it as much as possible.

Change-Id: I18fec19bf69e05a22a4142a9cd1165eccd022455
2017-05-23 08:35:15 +00:00
pangliye
2a8cb0b781 Modify the configuration file of telegraf
Influxdb is  standalone deployed by kolla-ansible,
The cluster mode is  charged now.
Previously configured, each telegraf monitors all influxdb.
If the telegraf is more, It is a disaster to influxdb.
So, telegraf should monitor influxdb deployed on the same server.

Change-Id: I9f4003a55de2a7be53c644b1d2f89d00cdf9478a
2017-05-18 07:37:16 +00:00
Jenkins
ed0f2fc819 Merge "Fix Telegraf retention policy not found" 2017-04-05 11:32:25 +00:00
Bertrand Lallau
bfecaf3857 Telegraf: activate influxdb input plugin
The InfluxDB plugin will collect metrics on the given InfluxDB servers.

Change-Id: I23f0433f56047369ce19bbf3236f6dd7038eb06e
2017-04-05 08:32:53 +02:00
Bertrand Lallau
a914fb6671 Fix Telegraf retention policy not found
New version of influxdb use "autogen" as default retention policy name.
Please see https://github.com/influxdata/influxdb/issues/3733 for more
info.

Change-Id: I8aeb47f60b3aeb022e0cd7aaac630d7cad5b0099
Closes-Bug: #1673914
2017-04-04 10:22:05 +02:00
Bertrand Lallau
8e3cce34a2 Fix Telegraf startup config files issue
Telegraf allow plugin configuration in /etc/telegraf/telegraf.d
directory. These files comes from telegraf/templates/config/*.conf,
but this directory is optional.

Closes-Bug: #1673691

Change-Id: Idc695c86c5f6b6b0bd514728f2025321cd200907
2017-03-20 22:47:31 +01:00
Nenad Radojevic
78c66cd992 Missing mysql plugin in telegraf
Added mysql plugin for telegraf to be used by default.
Plugin gathers the statistic data from MySQL server.

Change-Id: I853af61b64739eedb70ab2b8dc78f0285a839671
Closes-Bug: #1652609
2016-12-27 23:36:18 +00:00
Jenkins
86cd729c7d Merge "Remove inputs.filestat from telegraf configuration" 2016-11-06 04:58:26 +00:00
Christian Berendt
0c60cbe8de Collect haproxy/elasticsearch/memcached metrics only on related hosts
Change-Id: Ifa72d352e65c834afa078a7b08a7582155e6298b
Closes-bug: #1635032
2016-10-19 22:15:48 +02:00
Christian Berendt
c5791dbc29 Remove inputs.filestat from telegraf configuration
It makes no sense to monitor the filestats of local logfiles.

TrivialFix

Change-Id: I5ef060b57d2c88f922a05c59556ff5c96929f871
2016-10-18 18:45:45 +00:00
caoyuan
01310b8d7f Fix telegraf upgrade permission issue
Change-Id: If906f9a125f2fe11d1deb7eea5af4bcea3feac18
Partial-Bug: #1631503
2016-10-15 18:44:12 +08:00
Mathias Ewald
56af4ef202 Added telegraf role
Added ansible role to deploy telegraf

Roll out telegraf to all nodes

Introduce parameter to toggle deployment of performance monitoring

Co-Authored-By: zhubingbing <zhubingbing10@gmail.com>

Change-Id: Ia09b20ce65bf557c1a1030eda99df5cc88debd01
Partially-Implements: Blueprint performance-monitoring
2016-09-14 15:43:04 +00:00