Introduce kolla_address filter.
Introduce put_address_in_context filter.
Add AF config to vars.
Address contexts:
- raw (default): <ADDR>
- memcache: inet6:[<ADDR>]
- url: [<ADDR>]
Other changes:
globals.yml - mention just IP in comment
prechecks/port_checks (api_intf) - kolla_address handles validation
3x interface conditional (swift configs: replication/storage)
2x interface variable definition with hostname
(haproxy listens; api intf)
1x interface variable definition with hostname with bifrost exclusion
(baremetal pre-install /etc/hosts; api intf)
neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network
basic multinode source CI job for IPv6
prechecks for rabbitmq and qdrouterd use proper NSS database now
MariaDB Galera Cluster WSREP SST mariabackup workaround
(socat and IPv6)
Ceph naming workaround in CI
TODO: probably needs documenting
RabbitMQ IPv6-only proto_dist
Ceph ms switch to IPv6 mode
Remove neutron-server ml2_type_vxlan/vxlan_group setting
as it is not used (let's avoid any confusion)
and could break setups without proper multicast routing
if it started working (also IPv4-only)
haproxy upgrade checks for slaves based on ipv6 addresses
TODO:
ovs-dpdk grabs ipv4 network address (w/ prefix len / submask)
not supported, invalid by default because neutron_external has no address
No idea whether ovs-dpdk works at all atm.
ml2 for xenapi
Xen is not supported too well.
This would require working with XenAPI facts.
rp_filter setting
This would require meddling with ip6tables (there is no sysctl param).
By default nothing is dropped.
Unlikely we really need it.
ironic dnsmasq is configured IPv4-only
dnsmasq needs DHCPv6 options and testing in vivo.
KNOWN ISSUES (beyond us):
One cannot use IPv6 address to reference the image for docker like we
currently do, see: https://github.com/moby/moby/issues/39033
(docker_registry; docker API 400 - invalid reference format)
workaround: use hostname/FQDN
RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4.
This is due to old RabbitMQ versions available in images.
IPv4 is preferred by default and may fail in the IPv6-only scenario.
This should be no problem in real life as IPv6-only is indeed IPv6-only.
Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will
no longer be relevant as we supply all the necessary config.
See: https://github.com/rabbitmq/rabbitmq-server/pull/1982
For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed
to work well). Older Ansible versions are known to miss IPv6 addresses
in interface facts. This may affect redeploys, reconfigures and
upgrades which run after VIP address is assigned.
See: https://github.com/ansible/ansible/issues/63227
Bifrost Train does not support IPv6 deployments.
See: https://storyboard.openstack.org/#!/story/2006689
Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c
Implements: blueprint ipv6-control-plane
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
In the current deployment of ceph, the node name of osd and the name
of mon are both IP, and other daemons use hostname.
This commit adds support for naming mon and osd nodes using hostname,
and does not change the default ip-named way.
Change-Id: I22bef72dcd8fc8bcd391ae30e4643520250fd556
1) ceph-nfs (ganesha-ceph) - use NFSv4 only
This is recommended upstream.
v3 and UDP require portmapper (aka rpcbind) which we
do not want, except where Ubuntu ganesha version (2.6)
forces it by requiring enabled UDP, see [1].
The issue has been fixed in 2.8, included in CentOS.
Additionally disable v3 helper protocols and kerberos
to avoid meaningless warnings.
2) ceph-nfs (ganesha-ceph) - do not export host dbus
It is not in use. This avoids the temptation to try
handling it on host.
3) Properly handle ceph services deploy and upgrade
Upgrade runs deploy.
The order has been corrected - nfs goes after mds.
Additionally upgrade takes care of rgw for keystone
(for swift emulation).
4) Enhance ceph keyring module with error detection
Now it does not blindly try to create a keyring after
any failure. This used to hide real issue.
5) Retry ceph admin keyring update until cluster works
Reordering deployment caused issue with ceph cluster not being
fully operational before taking actions on it.
6) CI: Remove osd df from collected logs as it may hang CI
Hangs are caused by healthy MON and no healthy MGR.
A descriptive note is left in its place.
7) CI: Add 5s timeout to ceph informational commands
This decreases the timeout from the default 300s.
[1] https://review.opendev.org/669315
Change-Id: I1cf0ad10b80552f503898e723f0c4bd00a38f143
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
We're duplicating code to build the keystone URLs in nearly every
config, where we've already done it in group_vars. Replace the
redundancy with a variable that does the same thing.
Change-Id: I207d77870e2535c1cdcbc5eaf704f0448ac85a7a
when ceph_mon and ceph_osd start failed, add debug option will
print more info. now when ceph_mon and ceph_osd containers start
failed, docker logs ceph_mon print none log
Closes-Bug: #1815707
Change-Id: I3c5086019808a9738714f5279ec74cbb9b7a8587
when enable ceph_nfs,it deploy failed, because no ganesha config
file, and the 'ganesha.nfs' command need root privilege to run.
i will modify ceph_nfs dockerfile,please review. thanks
https://review.openstack.org/#/c/630510/
Change-Id: I347107bc33733061ad043bffe38ecc1d16770afc
Closes-Bug: #1811581
The buggy come from ceph changes[0], which is included since ceph osd
v11.0.0. The `osd crush update on start` logical is moved from
`ceph-osd-prestart.sh` to ceph-osd startup process. So ceph-osd will
create buckets by node hostname automatically. Whereas, kolla is
creating buckets by node ip
For the less confused and ceph upgrade impact, disable `osd crush update
on start` is a better choice
[0] a28b71e3c9
Change-Id: Ibbeac9505c9957319126267dbe6bd7a2cac11f0c
Closes-Bug: #1801662
ResellerAdmin role is used to give users object storage administration role
in their projects.
It is required to pass object storage quotas tests[1] of DefCore (OpenStack
Powered) certification test suite.
[1] tempest.api.object_storage.test_account_quotas*
Related-Bug: #1700729
Change-Id: Id976827aa7da271e54b77476f175f06bd1a00cc8
Currently test_list_containers tempest tests[1] would be failed.
It is becuase accept-ranges header does not exist. See ceph bug[2].
Rgw_swift_enforce_content_length assures Content-Length and
Accept-Ranges in dynamically generated account & container listings.
[1] tempest.api.object_storage.test_account_services.AccountTest.test_list_containers
[2] http://tracker.ceph.com/issues/21554
Related-Bug: #1783456
Change-Id: I9b5fcc361f0bc0e521302d2df1974aabf6f4a7e7
Object versioning test[1] is required for RefStack test suite.
Swift has enabled it by default[2].
It is also needed for ceph-rgw.
[1]
tempest.api.object_storage.test_object_version.ContainerTest.test_versioned_container
[2] https://review.openstack.org/#/c/517281/
Related-Bug: #1729583
Change-Id: If89636f77d87bab75e8e7bcf16cc784e83184bc6
By default ceph-rgw is not completely comaptible with Swift API,
because of the restriction for Swift INFO API.[0]
The patch improve ceph-rgw compatibility with Swift API. It is
controlled by the option "ceph_rgw_compatibility" in
ansible/group_vars/all.yml.
After changing the option, run the "reconfigure" command to enable.
Closes-Bug: #1783456
[0] https://github.com/ceph/ceph/pull/17967
Change-Id: Ibf3eb52280e197965caef08a44ae226c4f884cb5
Signed-off-by: tone.zhang <tone.zhang@arm.com>
For luminous, the ceph_mgr service provide the
dashborad for ceph, and it need connect to cluster,
but now it failed to connect ceph cluster due to
ceph.client.admin.keyring missing, this ps to fix it.
Co-Authored-By: chenqiaomin <chen.qiaomin@99cloud.net>
Closes-Bug: #1768462
Change-Id: Idb24661bc5674780db390220ca02e22975490663
ceph-mgr service is mandatory in ceph luminous
Depends-On: I875f84012a92d4f8b9dcb212d917cf61167270b8
Change-Id: I9418bf40a4bc3dcfc07c8b2eae17cb5779f5b444
Implements: blueprint ceph-luminous
kolla-kubernetes is using its own configuration generation[0], so it is
time for kolla-ansible to remove the related code to simplify the
logical.
[0] https://github.com/openstack/kolla-kubernetes/tree/master/ansible
Change-Id: I7bb0b7fe3b8eea906613e936d5e9d19f4f2e80bb
Implements: blueprint clean-k8s-config
There are to parameter for ceph-mon/ceph-osd/radosgw, "-f" and "-d".
-d run in foreground, log to stderr.
-f run in foreground, log to usual location.
When log tin stderr, docker engine will collect the logs. It is useless
and we should write the log into file only. Then stop writing log to
stderr.
ceph-mon log location is controlled by "mon cluster log file".
Change-Id: I7883a4316420bc2bf4c772cb7248b663359f54b7
Closes-Bug: #1677237
Default user group should be set much earlier in deployment
and should be used consistently accross all projects.
Change-Id: Id399f9ddebc903bb9c3eeb5a0ff6f33ca6d6828c
Closes-Bug: #1650501
This patch changes version of ceph from hammer to jewel. Also removed
versionlock as it seems we don't use it in ubuntu, and actually might be
risky if we miss security patch on ceph.
Change-Id: Ib8f88c2f914a4b635e59a509fa0194605eb73165
Implements: blueprint upgrade-ceph-to-jewel
Change the rgw frontend interface (from storage_interface to
api_interface) to make it consistent with the one specified in the
haproxy configuration.
Closes-Bug: #1603885
Change-Id: I8537f05575382faeac3ab915feaaad4af9b29565
The original commit changes ceph-rgw to listen on all
addresses (0.0.0.0:6780), which means that it listens on
both the storage_interface address and the VIP.
This makes it incompatible with having HAProxy on the same
host.
This reverts commit 819ae24b50ffda74e5b3a72f4c26321e7c9696cb.
Change-Id: I6cba31cc942a0be9f65d2f4847c8c2bc364a9c1e
Closes-Bug: 1596229
Signed-off-by: Dave Walker (Daviey) <email@daviey.com>
Fix 'rgw frontends' param on ceph-rgw container.
After apply this patch the service starts
http://paste.openstack.org/show/495786/
Closes-Bug: #1576817
Change-Id: I434ca2a72c035f457bd50ef5de20e62f89974e0c
These values are optional only when the services are not enabled.
If the file does not exist we should not warn, but rather inform.
Ceph-mon is an exception here since its bootstrap process means
the files may or may not exist initially.
TrivialFix
Change-Id: Ic02bece76d480e99deecf612036f37abb5604135
This was an attempt to get storage_interface to work properly but that
work will not be completed and functional this cycle. There are design
topics that need to be discussed about it that were brought to light
by the RAX gate failing for it.
TrivialFix
Change-Id: I65579f9e0e0dcf3fa51c0ea031ff474145457c40
Since the fetch script fetched _all_ keyrings from the ceph-mon
container, the ceph-mon container must contain all keyrings. This
setup works AIO but was broken on multinode because the ceph-mon
container did not have the radosgw keyring. This issue affects every
multinode install regardless of using the radosgw or not.
TrivialFix
Change-Id: Ie416de1a5275862da6d77ef0dd174e85e499fc0f
In heterogeneous environment, api_interfaces are different each other.
So we should specify it from hostvars.
Implements: bp configure-network-interface
Change-Id: Id15d70bfb9ebb62a64a3847a6b77407efb171dbe
This brings Kolla images inline with FHS and should make finding
locations of things more consistent and reliable with the linux world
at large.
Change-Id: Iece5b4da4bace0fb8b1f41a65ab2c852ec73e6f8
Closes-Bug: #1485742
Add the initial playbooks for making ceph and ansible play nice
together.
This does not include all of the openstack changes to make things like
nova, glance, and cinder work. This will simply build the ceph cluster
and thats it. The next patchset will do the OpenStack integration.
DocImpact
Change-Id: Ie1697dde5f92e833652933a80f0004f31b641330
Partially-Implements: blueprint ceph-container
