11 Commits

Author SHA1 Message Date
Clark Boylan
e5732b6137 Accomodate zuul's new key management system
Zuul is changing the way its key management system work from implicit
"backups" to explicit exports that can be used for backups. Additionally
to rename projects we will need to update those keys in zk which can be
done with copy and delete commands. We update the rename playbook to use
these.

Depends-On: https://review.opendev.org/c/zuul/zuul/+/803973
Change-Id: I2ba8015392f22ea615bcba7fb0d73a138dc77034
2021-08-09 14:45:49 -07:00
Clark Boylan
d4176aff5b Sync zuul status json backup list with current tenants
This removes the kata-containers tenant backup entry as that tenant no
longer exists. We also add status json backups for the opendev,
vexxhost, zuul, pyca, and pypa tenants. This gets us in sync with the
current tenant list.

Change-Id: I8527676dda67915e6ebe0d1c5fde7a57a7ac2e5b
2021-05-14 10:32:07 -07:00
Clark Boylan
39f10b5fe5 Fixup small issues on new zuul scheduler
This fixes the zuul debug log's logrotate filename. We also increase the
rotation count to 30 daily logs for all zuul scheduler zuul processes
(this matches the old server).

We also create a /var/lib/zuul/backup dir so that status.json backups
have a location they can write to. We do this in the base zuul role
which means all zuul servers will get this dir. It doesn't currently
conflict with any of the cluster members' /var/lib/zuul contents so
should be fine.

Change-Id: I4709e3c7e542781a65ae24c1f05a32444026fd26
2021-05-14 08:20:56 -07:00
James E. Blair
b6cbb52447 Add pull tasks for nodepool/zuul
So we can stop/pull/start, move the pull tasks to their own files
and add a playbook that invokes them.

Change-Id: I4f351c1d28e5e4606e0a778e545a3a805525ac71
2021-02-19 15:42:40 -08:00
James E. Blair
bfa60880a1 Remove old service cleanups from zuul
These cleanup tasks have all run and we no longer need to carry them.

Change-Id: I6130d1c2fbfe39ea339f1e18f3306b221b0e12e1
2021-01-22 15:51:06 -08:00
Monty Taylor
1832f8f060 Fix a few missing zuul_user usages
These got missed when we migrated everything else to zuuld.

Change-Id: Ia3dc9a1038622222e3bccd70e7dfe410b8bc89ca
2020-05-23 09:05:16 -05:00
Clark Boylan
c0fd3e0894 Pull and prune docker images together
We noticed that our zuul scheduler was running out of disk and one of
the causes of this is we are pulling all of the wonderful new zuul
images and not pruning them. This happens because we were only pruning
when (re)starting services and we don't do that automatically with Zuul.
Address this by always pruning after pulling even if we don't restart
services. This should be safe because prune will leave the latest tagged
images as well as the running images.

This should keep our disk consumption down.

Change-Id: Ibdd22ac42d86781f1e87c3d11e05fd8f99677167
2020-05-07 12:51:09 -07:00
Monty Taylor
c836437925 Remove old init scripts and services for zuul/nodepool
We're running these in containers now. Please not to try to start
them the old way.

failed_when false is because we can't disable the old service
in the gate if there is no service file installed.

Change-Id: Ia4560f385fc98e23f987a67a1dfa60c3188816b6
2020-05-06 17:13:58 -05:00
Clark Boylan
5acf91a3f6 Don't restart the zuul scheduler in prod
We use the zuul_scheduler_start flag to determine if we want to start
the zuul-scheduler when new containers show up. Unfortunately we weren't
setting zuul_scheduler_start in prod so we failed with this error:

  error while evaluating conditional (zuul_scheduler_start | bool): 'zuul_scheduler_start' is undefined

Fix this by treating an unset var as equivalent to a set truthy var
value. We do this instead of always setting the var to false in prod as
it simplifies testing.

Change-Id: I1f1a86e80199601646c7f2dec2a91c5d65d77231
2020-04-28 12:24:13 -07:00
Monty Taylor
5d1f556c2e Run smart-reconfigure instead of HUP
We don't want to HUP all the processes in the container, we just
want zuul to reconfigure. Use the smart-reconfigure command.

Also - start the scheduler in the gate job.

Change-Id: I66754ed168165d2444930ab1110e95316f7307a7
2020-04-26 11:01:08 -05:00
Monty Taylor
f0b77485ec Run Zuul using Ansible and Containers
Zuul is publishing lovely container images, so we should
go ahead and start using them.

We can't use containers for zuul-executor because of the
docker->bubblewrap->AFS issue, so install from pip there.

Don't start any of the containers by default, which should
let us safely roll this out and then do a rolling restart.
For things (like web or mergers) where it's safe to do so,
a followup change will swap the flag.

Change-Id: I37dcce3a67477ad3b2c36f2fd3657af18bc25c40
2020-04-24 09:18:44 -05:00