Make inventory/service for service-specific things, including the
groups.yaml group definitions, and inventory/base for hostvars
related to the base system, including the list of hosts.
Move the exisitng host_vars into inventory/service, since most of
them are likely service-specific. Move group_vars/all.yaml into
base/group_vars as almost all of it is related to base things,
with the execption of the gerrit public key.
A followup patch will move host-specific values into equivilent
files in inventory/base.
This should let us override hostvars in gate jobs. It should also
allow us to do better file matchers - and to be able to organize
our playbooks move if we want to.
Depends-On: https://review.opendev.org/731583
Change-Id: Iddf57b5be47c2e9de16b83a1bc83bee25db995cf
The time has come for me to step down my infra-root duties. Sadly, my
day to day job is no longer directly related to openstack-infra, and
finding it difficult to be involved in 'infra-root' capacity to help the
project.
Thanks to everything on the infra team, everybody is awesome humans! I
hope some time in the future I'll be able to get move involved with the
opendev.org effort, but sadly today isn't that day.
Change-Id: I986bc44f1a17ec76b5d7925b47eb65e6efbaad34
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
The existing test gearman cert+key combos were mismatched and therefore
invalid. This replaces them with newly generated test data, and moves
them into the test private hostvar files where the production private
data are now housed.
This removes the public production data as well; those certs are now
in the private hostvar files.
Change-Id: I6d7e12e2548f4c777854b8738c98f621bd10ad00
The jitsi video bridge (jvb) appears to be the main component we'll need
to scale up to handle more users on meetpad. Start preliminary
ansiblification of scale out jvb hosts.
Note this requires each new jvb to run on a separate host as the jvb
docker images seem to rely on $HOSTNAME to uniquely identify each jvb.
Change-Id: If6d055b6ec163d4a9d912bee9a9912f5a7b58125
This adds a new variable for the iptables role that allows us to
indicate all members of an ansible inventory group should have
iptables rules added.
It also removes the unused zuul-executor-opendev group, and some
unused variables related to the snmp rule.
Also, collect the generated iptables rules for debugging.
Change-Id: I48746a6527848a45a4debf62fd833527cc392398
Depends-On: https://review.opendev.org/728952
This avoids the conflict with the zuul user (1000) on the test
nodes. The executor will continue to use the default username
of 'zuul' as the ansible_user in the inventory.
This change also touches the zk and nodepool deployment to use
variables for the usernames and uids to make changes like this
easier. No changes are intended there.
Change-Id: Ib8cef6b7889b23ddc65a07bcba29c21a36e3dcb5
This autogenerates the list of ssl domains for the ssl-cert-check tool
directly from the letsencrypt list.
The first step is the install-certcheck role that replaces the
puppet-ssl_cert_check module that does the same. The reason for this
is so that during gate testing we can test this on the test
bridge.openstack.org server, and avoid adding another node as a
requirement for this test.
letsencrypt-request-certs is updated to set a fact
letsencrypt_certcheck_domains for each host that is generating a
certificate. As described in the comments, this defaults to the first
host specified for the certificate and the listening port can be
indicated (if set, this new port value is stripped when generating
certs as is not necessary for certificate generation).
The new letsencrypt-config-certcheck role runs and iterates all
letsencrypt hosts to build the final list of domains that should be
checked. This is then extended with the
letsencrypt_certcheck_additional_domains value that covers any hosts
using certificates not provisioned by letsencrypt using this
mechanism.
These additional domains are pre-populated from the openstack.org
domains in the extant check file, minus those openstack.org domain
certificates we are generating via letsencrypt (see
letsencrypt-create-certs/handlers/main.yaml). Additionally, we
update some of the certificate variables in host_vars that are
listening on port !443.
As mentioned, bridge.openstack.org is placed in the new certcheck
group for gate testing, so the tool and config file will be deployed
to it. For production, cacti is added to the group, which is where
the tool currently runs. The extant puppet installation is disabled,
pending removal in a follow-on change.
Change-Id: Idbe084f13f3684021e8efd9ac69b63fe31484606
This was missed in an earlier change where we enabled these vhosts.
Testing worked because testing was communicating to localhost and not
the public ip address.
This has been addressed as well.
Change-Id: I2d91aea466f1b587780a452cfe8e1396515930ed
Remove the separate "mirror_opendev" group and rename it to just
"mirror". Update various parts to reflect that change.
We no longer deploy any mirror hosts with puppet, remove the various
configuration files.
Depends-On: https://review.opendev.org/728345
Change-Id: Ia982fe9cb4357447989664f033df976b528aaf84
We want to replace the current executors with focal executors.
Make sure zuul-executor can run there.
Kubic is apparently the new source for libcontainers stuff:
https://podman.io/getting-started/installation.html
Use only timesyncd on focal
ntp and timesyncd have a hard conflict with each other. Our test
images install ntp. Remove it and just stay with timesyncd.
Change-Id: I0126f7c77d92deb91711f38a19384a9319955cf5
It takes over the log files. So does the sync of
project-config.
Depends-On: https://review.opendev.org/724418
Change-Id: Ic5c3811bf8b03cd387a2790e4d6ab457f5288c57
We have two standalone roles, puppet and cloud-launcher, but we
currently install them with galaxy so depends-on patches don't
work. We also install them every time we run anything, even if
we don't need them for the playbook in question.
Add two roles, one to install a set of ansible roles needed by
the host in question, and the other to encapsulate the sequence
of running puppet, which now includes installing the puppet
role, installing puppet, disabling the puppet agent and then
running puppet.
As a followup, we'll do the same thing with the puppet modules,
so that we arent' cloning and rsyncing ALL of the puppet modules
all the time no matter what.
Change-Id: I69a2e99e869ee39a3da573af421b18ad93056d5b
We get deprecation warnings from ansible about use
of python2 on xenial hosts. Rather than setting
ansible_python_interpreter to python3 on a host by
host basis, set it globally to python3.
Set it to python for the one host that's too old,
refstack.openstack.org, which is running on trusty
which only has python3.4.
Change-Id: I4965d950c13efad80d72912911bc7099e9da1659
Zuul is publishing lovely container images, so we should
go ahead and start using them.
We can't use containers for zuul-executor because of the
docker->bubblewrap->AFS issue, so install from pip there.
Don't start any of the containers by default, which should
let us safely roll this out and then do a rolling restart.
For things (like web or mergers) where it's safe to do so,
a followup change will swap the flag.
Change-Id: I37dcce3a67477ad3b2c36f2fd3657af18bc25c40
We run puppet with ansible now pretty much all the time. It's not
helpful for the puppet output to go to syslog on the remote host.
What's more helpful is for it to come back to the stdout in the
ansible playbook so that we can see it.
Also turn off ansi color from the output.
Depends-On: https://review.opendev.org/721732
Change-Id: I604081d5400bd53b8dda5a3a7685323c1443991b
Extract eavedrop into its own service playbook and
puppet manifest. While doing that, stop using jenkinsuser
on eavesdrop in favor of zuul-user.
Add the ability to override the keys for the zuul user.
Remove openstack_project::server, it doesn't do anything.
Containerize and anisblize accessbot. The structure of
how we're doing it in puppet makes it hard to actually
run the puppet in the gate. Run the script in its own
playbook so that we can avoid running it in the gate.
Change-Id: I53cb63ffa4ae50575d4fa37b24323ad13ec1bac3
We use project-config for gerrit, gitea and nodepool config. That's
cool, because can clone that from zuul too and make sure that each
prod run we're doing runs with the contents of the patch in question.
Introduce a flag file that can be touched in /home/zuulcd that will
block zuul from running prod playbooks. By default, if the file is
there, zuul will wait for an hour before giving up.
Rename zuulcd to zuul
To better align prod and test, name the zuul user zuul.
Change-Id: I83c38c9c430218059579f3763e02d6b9f40c7b89
We want to trigger nameserver updates when we merge patches
to zone files.
The zuul zone repo is currently managed by infra-core. We need to
make an improvement to zuul before we can offload core role there
to the zuul-maint team.
Change-Id: I6192f2499465844ccf2a1f903a8897458814da5d
We removed nb01 with I18ab9834ad4da201774e0abef56f618cd7839d36 and
replaced it with nb04; open the firewall for it.
Change-Id: I7138ee8744d978388b95e35ddd767cc97a5f5a87
As part of OpenDev rename, a lot of links were changed.
A couple of URLs point to old locations, update them.
This list was done while grepping for "openstack-infra" and fixing
locations that are wrong.
Change-Id: I313d76284bb549f1b2c636ce17fa662c233c0af9
Currently we deploy the openstacksdk config into ~nodepool/.config on
the container, and then map this directory back to /etc/openstack in
the docker-compose. The config-file still hard-codes the
limestone.pem file to ~nodepool/.config.
Switch the nodepool-builder_opendev group to install to
/etc/openstack, and update the nodepool config file template to use
the configured directory for the .pem path.
Also update the testing paths.
Story: #2007407
Task: #39015
Change-Id: I9ca77927046e2b2e3cee9a642d0bc566e3871515
This is a start at ansible-deployed nodepool environments.
We rename the minimal-nodepool element to nodepool-base-legacy, and
keep running that for the old nodes.
The groups are updated so that only the .openstack.org hosts will run
puppet. Essentially they should remain unchanged.
We start a nodepool-base element that will replace the current
puppet-<openstackci|nodepool> deployment parts. For step one, this
grabs project-config and links in the elements and config file.
A testing host is added for gate testing which should trigger these
roles. This will build into a full deployment test of the builder
container.
Change-Id: If0eb9f02763535bf200062c51a8a0f8793b1e1aa
Depends-On: https://review.opendev.org/#/c/710700/
I forgot this in some of the prior changes that moved afsmon and
afs-release.py to this host, and those jobs send stats.
Change-Id: Ifacf69e7fef5b54a03d43272e9cc01b6fbe8e845
We have a single vhost for zuul-ci.org and zuulci.org, so we should
request a cert with all 4 hostnames.
We also have a separate vhost to handle the git.zuul-ci.org redirect;
add a cert request for that so we can manage it with LE.
Change-Id: Ia2ba3d3ad4f5ab0356ede371d94af3c77a89eda1
I was trying to simplify things by having a restricted shell script
run by root. However, our base-setup called my bluff as we also need
to setup sshd to allow remote root logins from specific addresses.
It's looking easier to create a new user, and give it sudo permissions
to run the vos release script.
Change-Id: If70b27cb974eb8c1bafec2b7ef86d4f5cba3c4c5
I wasn't correctly sourcing the key; it has to come from hostvars as
it is in a different play on different hosts. This fixes it.
We also need to not have the base roles overwrite the authorized_keys
file each time. The key we provision can only run a limited script
that wraps "vos release".
Unfortunately our gitops falls down a bit here because we don't have
full testing for the AFS servers; put this on the todo list :) I have
run this manually for testing.
Change-Id: I0995434bde7e43082c01daa331c4b8b268d9b4bc
This change adds a proxy config for quay which should assist
us when gating using images provided by the publically
available registry.
Change-Id: I971705e59724e70bd9d42a6920cf4f883556f673
Signed-off-by: Kevin Carter <kecarter@redhat.com>
