opendev/system-config
Code Issues Proposed changes
99342db824
system-config/zuul.d/infra-prod.yaml

687 lines
21 KiB
YAML
Raw Normal View History

Refactor infra-prod jobs for parallel running Refactor the infra-prod jobs to specify dependencies so they can run in parallel. Change-Id: I8f6150ec2f696933c93560c11fed0fd16b11bf65
2021-10-21 14:43:48 +11:00
# NOTE: job dependencies keep this running in parallel and are defined
# in projects.yaml because it's easier to keep an overall view of
# what's happening in there.
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
# Make sure only one run of a system-config playbook happens at a time
- semaphore:
name: infra-prod-playbook
max: 1
- job:
name: infra-prod-playbook
Use infra-prod-base in infra-prod jobs This uses a new base job which handles pushing the git repos on to bridge since that must now happen in a trusted playbook. Depends-On: https://review.opendev.org/742934 Change-Id: Ie6d0668f83af801c0c0e920b676f2f49e19c59f6
2020-07-24 09:03:36 -07:00
parent: opendev-infra-prod-base
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
description: |
Run specified playbook against productions hosts.
This is a parent job designed to be inherited to enabled
CD deployment of our infrastructure. Set playbook_name to
specify the playbook relative to
/home/zuul/src/opendev.org/opendev/system-config/playbooks
Convert production playbooks to bastion host group Following-on from Iffb462371939989b03e5d6ac6c5df63aa7708513, instead of directly referring to a hostname when adding the bastion host to the inventory for the production playbooks, this finds it from the first element of the "bastion" group. As we do this twice for the run and post playbooks, abstract it into a role. The host value is currently "bridge.openstack.org" -- as is the existing hard-coding -- thus this is intended to be a no-op change. It is setting the foundation to make replacing the bastion host a simpler process in the future. Change-Id: I286796ebd71173019a627f8fe8d9a25d0bfc575a
2022-09-20 15:27:15 +10:00
on the bastion host.
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
abstract: true
Update Zuul job semaphore usage The semaphore property is deprecated; replaced with semaphores. Change-Id: Idc459e60b846a270d7d9a9eb3737391c4ce2dd17
2021-06-24 13:20:29 -07:00
semaphores: infra-prod-playbook
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
run: playbooks/zuul/run-production-playbook.yaml
production-playbook logs : move to post-run step If the production playbook times out, we don't get any logs collected with the run. By moving the log collection into a post-run step, we should always get something copied to help us diagnose what is going wrong. Change-Id: I3e99b80e442db0cc87f8e8c9728b7697a5e4d1d3
2022-07-14 16:32:58 +10:00
post-run: playbooks/zuul/run-production-playbook-post.yaml
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
required-projects:
- opendev/system-config
vars:
Double the default number of ansible forks We run these ansible jobs serially which means we don't gain much by forcing ansible to use a small number of forks. Double the default for our infra prod job fork count from 5 to 10 to see if this speeds up our deploy jobs. Note some jobs override this value to either add more forks or fewer when necessary. These are left as is. Change-Id: I6fded724cb9c8654153bcc5937eae7203326076e
2021-05-14 12:14:15 -07:00
infra_prod_ansible_forks: 10
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
infra_prod_playbook_collect_log: false
encrypt-logs: turn on for all prod playbooks We have validated that the log encryption/export path is working, so turn it on for all prod jobs. Change-Id: Ic04d5b6e716dffedc925cb799e3630027183d890
2022-02-24 09:57:55 +11:00
infra_prod_playbook_encrypt_log: true
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
nodeset:
nodes: []
- job:
Rename install-ansible to bootstrap-bridge This used to be called "bridge", but was then renamed with Ia7c8dd0e32b2c4aaa674061037be5ab66d9a3581 to install-ansible to be clearer. It is true that this is installing Ansible, but as part of our reworking for parallel jobs this is the also the synchronisation point where we should be deploying the system-config code to run for the buildset. Thus naming this "boostrap-bridge" should hopefully be clearer again about what's going on. I've added a note to the job calling out it's difference to the infra-prod-service-bridge job to hopefully also avoid some of the inital confusion. Change-Id: I4db1c883f237de5986edb4dc4c64860390cc8e22
2021-12-03 11:32:31 +11:00
name: infra-prod-bootstrap-bridge
infra-prod-bootstrap-bridge: run directly on bridge In discussion of other changes, I realised that the bridge bootstrap job is running via zuul/run-production-playbook.yaml. This means it uses the Ansible installed on bridge to run against itself -- which isn't much of a bootstrap. What should happen is that the bootstrap-bridge.yaml playbook, which sets up ansible and keys on the bridge node, should run directly from the executor against the bridge node. To achieve this we reparent the job to opendev-infra-prod-setup-keys, which sets up the executor to be able to log into the bridge node. We then add the host dynamically and run the bootstrap-bridge.yaml playbook against it. This is similar to the gate testing path; where bootstrap-bridge.yaml is run from the exeuctor against the ephemeral bridge testing node before the nested-Ansible is used. The root key deployment is updated to use the nested Ansible directly, so that it can read the variable from the on-host secrets. Change-Id: Iebaeed5028050d890ab541818f405978afd60124
2022-10-13 16:41:07 +11:00
parent: opendev-infra-prod-setup-keys
Rename install-ansible to bootstrap-bridge This used to be called "bridge", but was then renamed with Ia7c8dd0e32b2c4aaa674061037be5ab66d9a3581 to install-ansible to be clearer. It is true that this is installing Ansible, but as part of our reworking for parallel jobs this is the also the synchronisation point where we should be deploying the system-config code to run for the buildset. Thus naming this "boostrap-bridge" should hopefully be clearer again about what's going on. I've added a note to the job calling out it's difference to the infra-prod-service-bridge job to hopefully also avoid some of the inital confusion. Change-Id: I4db1c883f237de5986edb4dc4c64860390cc8e22
2021-12-03 11:32:31 +11:00
description: |
Configure the bastion host (bridge)
This job does minimal configuration on the bastion host
(bridge.openstack.org) to allow it to run system-config
infra-prod-bootstrap-bridge: run directly on bridge In discussion of other changes, I realised that the bridge bootstrap job is running via zuul/run-production-playbook.yaml. This means it uses the Ansible installed on bridge to run against itself -- which isn't much of a bootstrap. What should happen is that the bootstrap-bridge.yaml playbook, which sets up ansible and keys on the bridge node, should run directly from the executor against the bridge node. To achieve this we reparent the job to opendev-infra-prod-setup-keys, which sets up the executor to be able to log into the bridge node. We then add the host dynamically and run the bootstrap-bridge.yaml playbook against it. This is similar to the gate testing path; where bootstrap-bridge.yaml is run from the exeuctor against the ephemeral bridge testing node before the nested-Ansible is used. The root key deployment is updated to use the nested Ansible directly, so that it can read the variable from the on-host secrets. Change-Id: Iebaeed5028050d890ab541818f405978afd60124
2022-10-13 16:41:07 +11:00
playbooks against our production hosts. It sets up Ansible
and root keys on the host.
Rename install-ansible to bootstrap-bridge This used to be called "bridge", but was then renamed with Ia7c8dd0e32b2c4aaa674061037be5ab66d9a3581 to install-ansible to be clearer. It is true that this is installing Ansible, but as part of our reworking for parallel jobs this is the also the synchronisation point where we should be deploying the system-config code to run for the buildset. Thus naming this "boostrap-bridge" should hopefully be clearer again about what's going on. I've added a note to the job calling out it's difference to the infra-prod-service-bridge job to hopefully also avoid some of the inital confusion. Change-Id: I4db1c883f237de5986edb4dc4c64860390cc8e22
2021-12-03 11:32:31 +11:00
Note that this is separate to infra-prod-service-bridge;
bridge in it's role as the bastion host actaully runs that
against itself; it includes things not strictly needed to make
the host able to deploy system-config.
infra-prod-bootstrap-bridge: fix typo in playbook name Introduced with Iebaeed5028050d890ab541818f405978afd60124 Change-Id: I2e06221d03589dc6bcb5fb060b439e35e3d604dc
2022-10-19 11:10:21 +11:00
run: playbooks/zuul/run-production-bootstrap-bridge.yaml
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
files:
Rename install-ansible to bootstrap-bridge This used to be called "bridge", but was then renamed with Ia7c8dd0e32b2c4aaa674061037be5ab66d9a3581 to install-ansible to be clearer. It is true that this is installing Ansible, but as part of our reworking for parallel jobs this is the also the synchronisation point where we should be deploying the system-config code to run for the buildset. Thus naming this "boostrap-bridge" should hopefully be clearer again about what's going on. I've added a note to the job calling out it's difference to the infra-prod-service-bridge job to hopefully also avoid some of the inital confusion. Change-Id: I4db1c883f237de5986edb4dc4c64860390cc8e22
2021-12-03 11:32:31 +11:00
- playbooks/bootstrap-bridge.yaml
infra-prod-bootstrap-bridge: run directly on bridge In discussion of other changes, I realised that the bridge bootstrap job is running via zuul/run-production-playbook.yaml. This means it uses the Ansible installed on bridge to run against itself -- which isn't much of a bootstrap. What should happen is that the bootstrap-bridge.yaml playbook, which sets up ansible and keys on the bridge node, should run directly from the executor against the bridge node. To achieve this we reparent the job to opendev-infra-prod-setup-keys, which sets up the executor to be able to log into the bridge node. We then add the host dynamically and run the bootstrap-bridge.yaml playbook against it. This is similar to the gate testing path; where bootstrap-bridge.yaml is run from the exeuctor against the ephemeral bridge testing node before the nested-Ansible is used. The root key deployment is updated to use the nested Ansible directly, so that it can read the variable from the on-host secrets. Change-Id: Iebaeed5028050d890ab541818f405978afd60124
2022-10-13 16:41:07 +11:00
- playbooks/zuul/run-production-bootstrap-bridge.yaml
- playbooks/zuul/run-production-bootstrap-bridge-add-rootkey.yaml
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/install-ansible/
- playbooks/roles/root-keys/
Also bootstrap bridge any time inventory changes We need the infra-prod-bootstrap-bridge job to add SSH host keys from our Ansible inventory to the /etc/ssh_known_hosts on the bridge. When adding a new server to the inventory, any added host keys should be deployed. Make sure this happens. Change-Id: I422f80fc033cfe8e20d6d30b0fe23f82800c4cea
2022-11-29 20:48:23 +00:00
- inventory/base/hosts.yaml
Abstract name of bastion host for testing path This replaces hard-coding of the host "bridge.openstack.org" with hard-coding of the first (and only) host in the group "bastion". The idea here is that we can, as much as possible, simply switch one place to an alternative hostname for the bastion such as "bridge.opendev.org" when we upgrade. This is just the testing path, for now; a follow-on will modify the production path (which doesn't really get speculatively tested) This needs to be defined in two places : 1) We need to define this in the run jobs for Zuul to use in the playbooks/zuul/run-*.yaml playbooks, as it sets up and collects logs from the testing bastion host. 2) The nested Ansible run will then use inventory inventory/service/groups.yaml Various other places are updated to use this abstracted group as the bastion host. Variables are moved into the bastion group (which only has one host -- the actual bastion host) which means we only have to update the group mapping to the new host. This is intended to be a no-op change; all the jobs should work the same, but just using the new abstractions. Change-Id: Iffb462371939989b03e5d6ac6c5df63aa7708513
2022-09-20 11:57:22 +10:00
- inventory/service/group_vars/bastion.yaml
Run bootstrap-bridge with empty nodeset We are currently using the default nodeset on the infra-prod-bootstrap-bridge job which results in us waiting for a node that we end up ignoring. As far as I can tell this job runs against localhost and the add_host bridge entry. It ignores the default test node from the nodeset. Speed up job execution and reduce node waste by setting an empty nodeset on the job. Change-Id: I8c3ffda60b92a8655989579335a49423fbdd18a2
2023-08-17 09:59:09 -07:00
nodeset:
nodes: []
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- job:
name: infra-prod-base
parent: infra-prod-playbook
description: Run the base playbook everywhere.
vars:
playbook_name: base.yaml
infra_prod_ansible_forks: 50
files:
- inventory/
Split inventory into multiple dirs and move hostvars Make inventory/service for service-specific things, including the groups.yaml group definitions, and inventory/base for hostvars related to the base system, including the list of hosts. Move the exisitng host_vars into inventory/service, since most of them are likely service-specific. Move group_vars/all.yaml into base/group_vars as almost all of it is related to base things, with the execption of the gerrit public key. A followup patch will move host-specific values into equivilent files in inventory/base. This should let us override hostvars in gate jobs. It should also allow us to do better file matchers - and to be able to organize our playbooks move if we want to. Depends-On: https://review.opendev.org/731583 Change-Id: Iddf57b5be47c2e9de16b83a1bc83bee25db995cf
2020-05-26 15:46:41 -05:00
- inventory/service/host_vars/
- inventory/service/group_vars/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/base.yaml
Move base roles into a base subdir If we move these into a subdir, it cleans up the number of things we nave to files match on. Stop running disable-puppet-agent in base. We run it in run-puppet which should be fine. Change-Id: Ia16adb96b11d25a097490882c4c59a50a0b7b23d
2020-05-26 14:24:20 -05:00
- playbooks/roles/base/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- job:
Rename service-letsencrypt to just letsencrypt This isn't a service, it's a meta thing that we run for different hosts at different times. Change-Id: Ib65665c98afb3ddb94b15346931be88a4b1757d8
2020-05-28 16:43:14 -05:00
name: infra-prod-letsencrypt
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
parent: infra-prod-playbook
Rename service-letsencrypt to just letsencrypt This isn't a service, it's a meta thing that we run for different hosts at different times. Change-Id: Ib65665c98afb3ddb94b15346931be88a4b1757d8
2020-05-28 16:43:14 -05:00
description: Run letsencrypt.yaml playbook.
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
vars:
Rename service-letsencrypt to just letsencrypt This isn't a service, it's a meta thing that we run for different hosts at different times. Change-Id: Ib65665c98afb3ddb94b15346931be88a4b1757d8
2020-05-28 16:43:14 -05:00
playbook_name: letsencrypt.yaml
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
files:
- inventory/
Rename service-letsencrypt to just letsencrypt This isn't a service, it's a meta thing that we run for different hosts at different times. Change-Id: Ib65665c98afb3ddb94b15346931be88a4b1757d8
2020-05-28 16:43:14 -05:00
- playbooks/letsencrypt.yaml
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
# Any touching of host_vars or group_vars can substantively
# change the certs we're doing, so be greedy here.
Split inventory into multiple dirs and move hostvars Make inventory/service for service-specific things, including the groups.yaml group definitions, and inventory/base for hostvars related to the base system, including the list of hosts. Move the exisitng host_vars into inventory/service, since most of them are likely service-specific. Move group_vars/all.yaml into base/group_vars as almost all of it is related to base things, with the execption of the gerrit public key. A followup patch will move host-specific values into equivilent files in inventory/base. This should let us override hostvars in gate jobs. It should also allow us to do better file matchers - and to be able to organize our playbooks move if we want to. Depends-On: https://review.opendev.org/731583 Change-Id: Iddf57b5be47c2e9de16b83a1bc83bee25db995cf
2020-05-26 15:46:41 -05:00
- inventory/service/host_vars/
- inventory/service/group_vars/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/letsencrypt
- playbooks/roles/logrotate/
- job:
name: infra-prod-manage-projects
parent: infra-prod-playbook
Bump the infra-prod-manage-projects job timeout Bump this timeout for a couple of reasons. First we've seen the job timeout at least once in the last month. This seems to be due to gitea portions of the job running slowly. Second we're planning some large scale updates to the openstack acls and a longer timeout should help us get those in in larger batches. We can consider trimming this back again after these updates are done if gitea doesn't continue to give us trouble. Change-Id: Ib61849b4c73a1b3fa2a0bbe90ace29fb23849449
2021-04-16 14:10:34 -07:00
timeout: 4800
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
description: |
Create and update projects in gerrit and gitea.
allowed-projects:
- opendev/system-config
- openstack/project-config
required-projects:
- opendev/system-config
- openstack/project-config
vars:
playbook_name: manage-projects.yaml
infra_prod_ansible_forks: 10
infra_prod_playbook_collect_log: true
- job:
name: infra-prod-service-base
parent: infra-prod-playbook
description: Base job for most service playbooks.
abstract: true
- job:
name: infra-prod-service-bridge
parent: infra-prod-service-base
description: Run service-bridge.yaml playbook.
vars:
playbook_name: service-bridge.yaml
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/service-bridge.yaml
Abstract name of bastion host for testing path This replaces hard-coding of the host "bridge.openstack.org" with hard-coding of the first (and only) host in the group "bastion". The idea here is that we can, as much as possible, simply switch one place to an alternative hostname for the bastion such as "bridge.opendev.org" when we upgrade. This is just the testing path, for now; a follow-on will modify the production path (which doesn't really get speculatively tested) This needs to be defined in two places : 1) We need to define this in the run jobs for Zuul to use in the playbooks/zuul/run-*.yaml playbooks, as it sets up and collects logs from the testing bastion host. 2) The nested Ansible run will then use inventory inventory/service/groups.yaml Various other places are updated to use this abstracted group as the bastion host. Variables are moved into the bastion group (which only has one host -- the actual bastion host) which means we only have to update the group mapping to the new host. This is intended to be a no-op change; all the jobs should work the same, but just using the new abstractions. Change-Id: Iffb462371939989b03e5d6ac6c5df63aa7708513
2022-09-20 11:57:22 +10:00
- inventory/service/group_vars/bastion.yaml
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/logrotate/
- playbooks/roles/edit-secrets-script/
- playbooks/roles/install-kubectl/
Split inventory into multiple dirs and move hostvars Make inventory/service for service-specific things, including the groups.yaml group definitions, and inventory/base for hostvars related to the base system, including the list of hosts. Move the exisitng host_vars into inventory/service, since most of them are likely service-specific. Move group_vars/all.yaml into base/group_vars as almost all of it is related to base things, with the execption of the gerrit public key. A followup patch will move host-specific values into equivilent files in inventory/base. This should let us override hostvars in gate jobs. It should also allow us to do better file matchers - and to be able to organize our playbooks move if we want to. Depends-On: https://review.opendev.org/731583 Change-Id: Iddf57b5be47c2e9de16b83a1bc83bee25db995cf
2020-05-26 15:46:41 -05:00
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/configure-kubectl/
- playbooks/roles/configure-openstacksdk/
- playbooks/templates/clouds/bridge_all_clouds.yaml.j2
- job:
name: infra-prod-service-gitea-lb
parent: infra-prod-service-base
description: Run service-gitea-lb.yaml playbook.
vars:
playbook_name: service-gitea-lb.yaml
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/service-gitea-lb.yaml
Split inventory into multiple dirs and move hostvars Make inventory/service for service-specific things, including the groups.yaml group definitions, and inventory/base for hostvars related to the base system, including the list of hosts. Move the exisitng host_vars into inventory/service, since most of them are likely service-specific. Move group_vars/all.yaml into base/group_vars as almost all of it is related to base things, with the execption of the gerrit public key. A followup patch will move host-specific values into equivilent files in inventory/base. This should let us override hostvars in gate jobs. It should also allow us to do better file matchers - and to be able to organize our playbooks move if we want to. Depends-On: https://review.opendev.org/731583 Change-Id: Iddf57b5be47c2e9de16b83a1bc83bee25db995cf
2020-05-26 15:46:41 -05:00
- inventory/service/group_vars/gitea-lb.yaml
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/pip3/
Run iptables in service playbooks instead of base It's the only part of base that's important to run when we run a service. Run it in the service playbooks and get rid of the dependency on infra-prod-base. Continue running it in base so that new nodes are brought up with iptables in place. Bump the timeout for the mirror job, because the iptables addition seems to have just bumped it over the edge. Change-Id: I4608216f7a59cfa96d3bdb191edd9bc7bb9cca39
2020-05-26 17:41:54 -05:00
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/install-docker/
- playbooks/roles/haproxy/
- job:
name: infra-prod-service-nameserver
parent: infra-prod-service-base
description: Run service-nameserver.yaml playbook.
vars:
playbook_name: service-nameserver.yaml
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/service-nameserver.yaml
Split inventory into multiple dirs and move hostvars Make inventory/service for service-specific things, including the groups.yaml group definitions, and inventory/base for hostvars related to the base system, including the list of hosts. Move the exisitng host_vars into inventory/service, since most of them are likely service-specific. Move group_vars/all.yaml into base/group_vars as almost all of it is related to base things, with the execption of the gerrit public key. A followup patch will move host-specific values into equivilent files in inventory/base. This should let us override hostvars in gate jobs. It should also allow us to do better file matchers - and to be able to organize our playbooks move if we want to. Depends-On: https://review.opendev.org/731583 Change-Id: Iddf57b5be47c2e9de16b83a1bc83bee25db995cf
2020-05-26 15:46:41 -05:00
- inventory/service/group_vars/adns.yaml
Refactor adns variables Firstly, my understanding of "adns" is that it's short for authoritative-dns; i.e. things related to our main non-recursive DNS servers for the zones we manage. The "a" is useful to distinguish this from any sort of other dns services we might run for CI, etc. The way we do this is with a "hidden" server that applies updates from config management, which then notifies secondary public servers which do a zone transfer from the primary. They're all "authoritative" in the sense they're not for general recursive queries. As mentioned in Ibd8063e92ad7ff9ee683dcc7dfcc115a0b19dcaa, we currently have 3 groups adns : the hidden primary bind server ns : the secondary public authoratitive servers dns : both of the above This proposes a refactor into the following 3 groups adns-primary : hidden primary bind server adns-secondary : the secondary public authoritative servers adns : both of the above This is meant to be a no-op; I just feel like this makes it a bit clearer as to the "lay of the land" with these servers. It will need some considering of the hiera variables on bridge if we merge. Change-Id: I9ffef52f27bd23ceeec07fe0f45f9fee08b5559a
2023-03-09 15:01:45 +11:00
- inventory/service/group_vars/adns-primary.yaml
- inventory/service/group_vars/adns-secondary.yaml
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/master-nameserver/
- playbooks/roles/nameserver/
Run iptables in service playbooks instead of base It's the only part of base that's important to run when we run a service. Run it in the service playbooks and get rid of the dependency on infra-prod-base. Continue running it in base so that new nodes are brought up with iptables in place. Bump the timeout for the mirror job, because the iptables addition seems to have just bumped it over the edge. Change-Id: I4608216f7a59cfa96d3bdb191edd9bc7bb9cca39
2020-05-26 17:41:54 -05:00
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- job:
name: infra-prod-service-nodepool
parent: infra-prod-service-base
description: Run service-nodepool.yaml playbook.
vars:
playbook_name: service-nodepool.yaml
required-projects:
- opendev/system-config
- openstack/project-config
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/service-nodepool.yaml
Split inventory into multiple dirs and move hostvars Make inventory/service for service-specific things, including the groups.yaml group definitions, and inventory/base for hostvars related to the base system, including the list of hosts. Move the exisitng host_vars into inventory/service, since most of them are likely service-specific. Move group_vars/all.yaml into base/group_vars as almost all of it is related to base things, with the execption of the gerrit public key. A followup patch will move host-specific values into equivilent files in inventory/base. This should let us override hostvars in gate jobs. It should also allow us to do better file matchers - and to be able to organize our playbooks move if we want to. Depends-On: https://review.opendev.org/731583 Change-Id: Iddf57b5be47c2e9de16b83a1bc83bee25db995cf
2020-05-26 15:46:41 -05:00
- inventory/service/host_vars/nb
- inventory/service/host_vars/nl
- inventory/service/group_vars/nodepool
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/configure-kubectl/
- playbooks/roles/configure-openstacksdk/
- playbooks/roles/install-docker/
Run iptables in service playbooks instead of base It's the only part of base that's important to run when we run a service. Run it in the service playbooks and get rid of the dependency on infra-prod-base. Continue running it in base so that new nodes are brought up with iptables in place. Bump the timeout for the mirror job, because the iptables addition seems to have just bumped it over the edge. Change-Id: I4608216f7a59cfa96d3bdb191edd9bc7bb9cca39
2020-05-26 17:41:54 -05:00
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/nodepool
- playbooks/templates/clouds/nodepool_
- job:
name: infra-prod-service-etherpad
parent: infra-prod-service-base
description: Run service-etherpad.yaml playbook.
vars:
playbook_name: service-etherpad.yaml
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/service-etherpad.yaml
Make etherpad configuration more generic for multiple hosts This switches us to running the services against the etherpad group. We also define vars in a group_vars file rather than a host specific file. This allows us to switch testing over to etherpad99 to decouple it from our production hostnames. A followup change will add a new etherpad production server that will be deployed alongside the existing one. This refactor makes that a bit simpler. Change-Id: I838ad31eb74a3abfd02bbfa77c9c2d007d57a3d4
2023-04-03 14:56:17 -07:00
- inventory/service/group_vars/etherpad.yaml
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/install-docker/
- playbooks/roles/pip3/
- playbooks/roles/etherpad
- playbooks/roles/logrotate
Run iptables in service playbooks instead of base It's the only part of base that's important to run when we run a service. Run it in the service playbooks and get rid of the dependency on infra-prod-base. Continue running it in base so that new nodes are brought up with iptables in place. Bump the timeout for the mirror job, because the iptables addition seems to have just bumped it over the edge. Change-Id: I4608216f7a59cfa96d3bdb191edd9bc7bb9cca39
2020-05-26 17:41:54 -05:00
- playbooks/roles/iptables/
Run our etherpad prod deploy job when docker updates We want to pick up changes to our docker setup in production. Without this we don't get the infra-prod-service-etherpad job running when we update the etherpad docker image. Change-Id: I25aee457b7c0547fc11439301054bb5aef799476
2020-07-17 13:20:48 -07:00
- docker/etherpad/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
Add a keycloak server This adds a keycloak server so we can start experimenting with it. It's based on the docker-compose file Matthieu made for Zuul (see https://review.opendev.org/819745 ) We should be able to configure a realm and federate with openstackid and other providers as described in the opendev auth spec. However, I am unable to test federation with openstackid due its inability to configure an oauth app at "localhost". Therefore, we will need an actual deployed system to test it. This should allow us to do so. It will also allow use to connect realms to the newly available Zuul admin api on opendev. It should be possible to configure the realm the way we want, then export its configuration into a JSON file and then have our playbooks or the docker-compose file import it. That would allow us to drive change to the configuration of the system through code review. Because of the above limitation with openstackid, I think we should regard the current implementation as experimental. Once we have a realm configuration that we like (which we will create using the GUI), we can chose to either continue to maintain the config with the GUI and appropriate file backups, or switch to a gitops model based on an export. My understanding is that all the data (realms configuration and session) are kept in an H2 database. This is probably sufficient for now and even production use with Zuul, but we should probably switch to mariadb before any heavy (eg gerrit, etc) production use. This is a partial implementation of https://docs.opendev.org/opendev/infra-specs/latest/specs/central-auth.html We can re-deploy with a new domain when it exists. Change-Id: I2e069b1b220dbd3e0a5754ac094c2b296c141753 Co-Authored-By: Matthieu Huin <mhuin@redhat.com>
2021-11-30 13:03:12 -08:00
- job:
name: infra-prod-service-keycloak
parent: infra-prod-service-base
description: Run service-keycloak.yaml playbook.
vars:
playbook_name: service-keycloak.yaml
files:
- inventory/base
- playbooks/service-keycloak.yaml
Upgrade to Keycloak 23.0 This includes a switch from the "legacy" style Wildfly-based image to a new setup using Quarkus. Because Keycloak maintainers consider H2 databases as a test/dev only option, there are no good migration and upgrade paths short of export/import data. Go ahead and change our deployment model to rely on a proper RDBMS, run locally from a container on the same server. Change-Id: I01f8045563e9f6db6168b92c5a868b8095c0d97b
2024-01-29 19:23:13 +00:00
- inventory/service/group_vars/keycloak.yaml
Add a keycloak server This adds a keycloak server so we can start experimenting with it. It's based on the docker-compose file Matthieu made for Zuul (see https://review.opendev.org/819745 ) We should be able to configure a realm and federate with openstackid and other providers as described in the opendev auth spec. However, I am unable to test federation with openstackid due its inability to configure an oauth app at "localhost". Therefore, we will need an actual deployed system to test it. This should allow us to do so. It will also allow use to connect realms to the newly available Zuul admin api on opendev. It should be possible to configure the realm the way we want, then export its configuration into a JSON file and then have our playbooks or the docker-compose file import it. That would allow us to drive change to the configuration of the system through code review. Because of the above limitation with openstackid, I think we should regard the current implementation as experimental. Once we have a realm configuration that we like (which we will create using the GUI), we can chose to either continue to maintain the config with the GUI and appropriate file backups, or switch to a gitops model based on an export. My understanding is that all the data (realms configuration and session) are kept in an H2 database. This is probably sufficient for now and even production use with Zuul, but we should probably switch to mariadb before any heavy (eg gerrit, etc) production use. This is a partial implementation of https://docs.opendev.org/opendev/infra-specs/latest/specs/central-auth.html We can re-deploy with a new domain when it exists. Change-Id: I2e069b1b220dbd3e0a5754ac094c2b296c141753 Co-Authored-By: Matthieu Huin <mhuin@redhat.com>
2021-11-30 13:03:12 -08:00
- playbooks/roles/keycloak/
- playbooks/roles/install-docker/
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- job:
name: infra-prod-service-meetpad
parent: infra-prod-service-base
description: Run service-meetpad.yaml playbook.
vars:
playbook_name: service-meetpad.yaml
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/service-meetpad.yaml
Split inventory into multiple dirs and move hostvars Make inventory/service for service-specific things, including the groups.yaml group definitions, and inventory/base for hostvars related to the base system, including the list of hosts. Move the exisitng host_vars into inventory/service, since most of them are likely service-specific. Move group_vars/all.yaml into base/group_vars as almost all of it is related to base things, with the execption of the gerrit public key. A followup patch will move host-specific values into equivilent files in inventory/base. This should let us override hostvars in gate jobs. It should also allow us to do better file matchers - and to be able to organize our playbooks move if we want to. Depends-On: https://review.opendev.org/731583 Change-Id: Iddf57b5be47c2e9de16b83a1bc83bee25db995cf
2020-05-26 15:46:41 -05:00
- inventory/service/host_vars/meetpad01.opendev.org.yaml
- inventory/service/group_vars/meetpad.yaml
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/pip3/
- playbooks/roles/install-docker/
Run iptables in service playbooks instead of base It's the only part of base that's important to run when we run a service. Run it in the service playbooks and get rid of the dependency on infra-prod-base. Continue running it in base so that new nodes are brought up with iptables in place. Bump the timeout for the mirror job, because the iptables addition seems to have just bumped it over the edge. Change-Id: I4608216f7a59cfa96d3bdb191edd9bc7bb9cca39
2020-05-26 17:41:54 -05:00
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/jitsi-meet/
- job:
name: infra-prod-service-mirror-update
parent: infra-prod-service-base
description: Run service-mirror-update.yaml playbook.
vars:
playbook_name: service-mirror-update.yaml
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
- inventory/service/group_vars/mirror.yaml
- inventory/service/host_vars/mirror
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/service-mirror-update.yaml
- playbooks/roles/mirror-update/
reprepro: run deploy job on role changes This was missed so the deploy job doesn't run automatically on changes to the reprepro role. Change-Id: I8e40926bd5bbb2836c63704ba02a43e45c12b4c4
2020-10-27 16:29:48 +11:00
- playbooks/roles/reprepro/
Run iptables in service playbooks instead of base It's the only part of base that's important to run when we run a service. Run it in the service playbooks and get rid of the dependency on infra-prod-base. Continue running it in base so that new nodes are brought up with iptables in place. Bump the timeout for the mirror job, because the iptables addition seems to have just bumped it over the edge. Change-Id: I4608216f7a59cfa96d3bdb191edd9bc7bb9cca39
2020-05-26 17:41:54 -05:00
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/logrotate/
Fix up openafs-client job matching The jobs should have file matchers for "roles/openafs-client" (not playbooks/). Fix this. Add the openafs/kerberos role matchers to Zuul as well, as it uses them on the executors. Change-Id: I66fd7792d6b533362606291e1bfc01dfa2a2e05b
2021-03-03 13:41:56 +11:00
- roles/kerberos-client/
- roles/openafs-client/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- job:
name: infra-prod-service-mirror
parent: infra-prod-service-base
description: Run service-mirror.yaml playbook.
vars:
playbook_name: service-mirror.yaml
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/service-mirror.yaml
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/service/host_vars/mirror
Split inventory into multiple dirs and move hostvars Make inventory/service for service-specific things, including the groups.yaml group definitions, and inventory/base for hostvars related to the base system, including the list of hosts. Move the exisitng host_vars into inventory/service, since most of them are likely service-specific. Move group_vars/all.yaml into base/group_vars as almost all of it is related to base things, with the execption of the gerrit public key. A followup patch will move host-specific values into equivilent files in inventory/base. This should let us override hostvars in gate jobs. It should also allow us to do better file matchers - and to be able to organize our playbooks move if we want to. Depends-On: https://review.opendev.org/731583 Change-Id: Iddf57b5be47c2e9de16b83a1bc83bee25db995cf
2020-05-26 15:46:41 -05:00
- inventory/service/group_vars/mirror.yaml
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/mirror/
- playbooks/roles/afs-release/
- playbooks/roles/afsmon/
Run iptables in service playbooks instead of base It's the only part of base that's important to run when we run a service. Run it in the service playbooks and get rid of the dependency on infra-prod-base. Continue running it in base so that new nodes are brought up with iptables in place. Bump the timeout for the mirror job, because the iptables addition seems to have just bumped it over the edge. Change-Id: I4608216f7a59cfa96d3bdb191edd9bc7bb9cca39
2020-05-26 17:41:54 -05:00
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/logrotate/
Fix up openafs-client job matching The jobs should have file matchers for "roles/openafs-client" (not playbooks/). Fix this. Add the openafs/kerberos role matchers to Zuul as well, as it uses them on the executors. Change-Id: I66fd7792d6b533362606291e1bfc01dfa2a2e05b
2021-03-03 13:41:56 +11:00
- roles/openafs-client/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
Add paste service The paste service needs an upgrade; since others have created a lodgeit container it seems worth us keeping the service going if only to maintain the historical corpus of pastes. This adds the ansible to deploy lodgeit and a sibling mariadb container. I have imported a dump of the old data as a test. The dump is ~4gb and imported it takes up about double that; certainly nothing we need to be too concerned over. The server will be more than capable of running the db container alongside the lodgeit instance. This should have no effect on production until we decide to switch DNS. Change-Id: I284864217aa49d664ddc3ebdc800383b2d7e00e3
2021-06-29 11:57:10 +10:00
- job:
name: infra-prod-service-paste
parent: infra-prod-service-base
description: Run service-paste.yaml playbook.
vars:
playbook_name: service-paste.yaml
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
Add paste service The paste service needs an upgrade; since others have created a lodgeit container it seems worth us keeping the service going if only to maintain the historical corpus of pastes. This adds the ansible to deploy lodgeit and a sibling mariadb container. I have imported a dump of the old data as a test. The dump is ~4gb and imported it takes up about double that; certainly nothing we need to be too concerned over. The server will be more than capable of running the db container alongside the lodgeit instance. This should have no effect on production until we decide to switch DNS. Change-Id: I284864217aa49d664ddc3ebdc800383b2d7e00e3
2021-06-29 11:57:10 +10:00
- playbooks/service-paste.yaml
paste: move certificate to group variable Similar to Id98768e29a06cebaf645eb75b39e4dc5adb8830d, move the certificate variables to the group definition file, so that we don't have to duplicate handlers or definitions for the testing host. Change-Id: I6650f5621a4969582f40700232a596d84e2b4a06
2022-07-07 10:58:50 +10:00
- inventory/service/group_vars/paste.yaml
Add paste service The paste service needs an upgrade; since others have created a lodgeit container it seems worth us keeping the service going if only to maintain the historical corpus of pastes. This adds the ansible to deploy lodgeit and a sibling mariadb container. I have imported a dump of the old data as a test. The dump is ~4gb and imported it takes up about double that; certainly nothing we need to be too concerned over. The server will be more than capable of running the db container alongside the lodgeit instance. This should have no effect on production until we decide to switch DNS. Change-Id: I284864217aa49d664ddc3ebdc800383b2d7e00e3
2021-06-29 11:57:10 +10:00
- playbooks/roles/install-docker/
- playbooks/roles/pip3/
- playbooks/roles/lodgeit/
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- job:
name: infra-prod-service-static
parent: infra-prod-service-base
description: Run service-static.yaml playbook.
vars:
playbook_name: service-static.yaml
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/service-static.yaml
Split inventory into multiple dirs and move hostvars Make inventory/service for service-specific things, including the groups.yaml group definitions, and inventory/base for hostvars related to the base system, including the list of hosts. Move the exisitng host_vars into inventory/service, since most of them are likely service-specific. Move group_vars/all.yaml into base/group_vars as almost all of it is related to base things, with the execption of the gerrit public key. A followup patch will move host-specific values into equivilent files in inventory/base. This should let us override hostvars in gate jobs. It should also allow us to do better file matchers - and to be able to organize our playbooks move if we want to. Depends-On: https://review.opendev.org/731583 Change-Id: Iddf57b5be47c2e9de16b83a1bc83bee25db995cf
2020-05-26 15:46:41 -05:00
- inventory/service/group_vars/static.yaml
Run gitea and static tests when update Apache UA filters The gitea and static services both deploy apache using our UA filters. We were not testing these services when updating UA filters. This change fixes that giving us some basic sanity checking that UA filter updates are functional and quicker deployments of these filters. Change-Id: Icbe6558bb47946299a43905e2f64522576bad939
2023-05-02 10:16:29 -07:00
- playbooks/roles/apache-ua-filter/
Run iptables in service playbooks instead of base It's the only part of base that's important to run when we run a service. Run it in the service playbooks and get rid of the dependency on infra-prod-base. Continue running it in base so that new nodes are brought up with iptables in place. Bump the timeout for the mirror job, because the iptables addition seems to have just bumped it over the edge. Change-Id: I4608216f7a59cfa96d3bdb191edd9bc7bb9cca39
2020-05-26 17:41:54 -05:00
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/static/
- playbooks/roles/zuul-user/
Fix up openafs-client job matching The jobs should have file matchers for "roles/openafs-client" (not playbooks/). Fix this. Add the openafs/kerberos role matchers to Zuul as well, as it uses them on the executors. Change-Id: I66fd7792d6b533362606291e1bfc01dfa2a2e05b
2021-03-03 13:41:56 +11:00
- roles/openafs-client/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
Add Jaeger tracing server Change-Id: I1aa68b1d5f99364fa09776301894b922ed169a3a
2022-09-05 13:42:18 -07:00
- job:
name: infra-prod-service-tracing
parent: infra-prod-service-base
description: Run service-tracing.yaml playbook.
vars:
playbook_name: service-tracing.yaml
files:
- inventory/base
- playbooks/service-tracing.yaml
- inventory/service/group_vars/tracing.yaml
- playbooks/roles/jaeger/
- playbooks/roles/install-docker/
- playbooks/roles/iptables/
Add borg-backup roles This adds roles to implement backup with borg [1]. Our current tool "bup" has no Python 3 support and is not packaged for Ubuntu Focal. This means it is effectively end-of-life. borg fits our model of servers backing themselves up to a central location, is well documented and seems well supported. It also has the clarkb seal of approval :) As mentioned, borg works in the same manner as bup by doing an efficient back up over ssh to a remote server. The core of these roles are the same as the bup based ones; in terms of creating a separate user for each host and deploying keys and ssh config. This chooses to install borg in a virtualenv on /opt. This was chosen for a number of reasons; firstly reading the history of borg there have been incompatible updates (although they provide a tool to update repository formats); it seems important that we both pin the version we are using and keep clients and server in sync. Since we have a hetrogenous distribution collection we don't want to rely on the packaged tools which may differ. I don't feel like this is a great application for a container; we actually don't want it that isolated from the base system because it's goal is to read and copy it offsite with as little chance of things going wrong as possible. Borg has a lot of support for encrypting the data at rest in various ways. However, that introduces the possibility we could lose both the key and the backup data. Really the only thing stopping this is key management, and if we want to go down this path we can do it as a follow-on. The remote end server is configured via ssh command rules to run in append-only mode. This means a misbehaving client can't delete its old backups. In theory we can prune backups on the server side -- something we could not do with bup. The documentation has been updated but is vague on this part; I think we should get some hosts in operation, see how the de-duplication is working out and then decide how we want to mange things long term. Testing is added; a focal and bionic host both run a full backup of themselves to the backup server. Pretty cool, the logs are in /var/log/borg-backup-<host>.log. No hosts are currently in the borg groups, so this can be applied without affecting production. I'd suggest the next steps are to bring up a borg-based backup server and put a few hosts into this. After running for a while, we can add all hosts, and then deprecate the current bup-based backup server in vexxhost and replace that with a borg-based one; giving us dual offsite backups. [1] https://borgbackup.readthedocs.io/en/stable/ Change-Id: I2a125f2fac11d8e3a3279eb7fa7adb33a3acaa4e
2020-07-16 13:43:18 +10:00
- job:
name: infra-prod-service-borg-backup
parent: infra-prod-service-base
description: Run service-borg-backup.yaml playbook.
vars:
playbook_name: service-borg-backup.yaml
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
Add borg-backup roles This adds roles to implement backup with borg [1]. Our current tool "bup" has no Python 3 support and is not packaged for Ubuntu Focal. This means it is effectively end-of-life. borg fits our model of servers backing themselves up to a central location, is well documented and seems well supported. It also has the clarkb seal of approval :) As mentioned, borg works in the same manner as bup by doing an efficient back up over ssh to a remote server. The core of these roles are the same as the bup based ones; in terms of creating a separate user for each host and deploying keys and ssh config. This chooses to install borg in a virtualenv on /opt. This was chosen for a number of reasons; firstly reading the history of borg there have been incompatible updates (although they provide a tool to update repository formats); it seems important that we both pin the version we are using and keep clients and server in sync. Since we have a hetrogenous distribution collection we don't want to rely on the packaged tools which may differ. I don't feel like this is a great application for a container; we actually don't want it that isolated from the base system because it's goal is to read and copy it offsite with as little chance of things going wrong as possible. Borg has a lot of support for encrypting the data at rest in various ways. However, that introduces the possibility we could lose both the key and the backup data. Really the only thing stopping this is key management, and if we want to go down this path we can do it as a follow-on. The remote end server is configured via ssh command rules to run in append-only mode. This means a misbehaving client can't delete its old backups. In theory we can prune backups on the server side -- something we could not do with bup. The documentation has been updated but is vague on this part; I think we should get some hosts in operation, see how the de-duplication is working out and then decide how we want to mange things long term. Testing is added; a focal and bionic host both run a full backup of themselves to the backup server. Pretty cool, the logs are in /var/log/borg-backup-<host>.log. No hosts are currently in the borg groups, so this can be applied without affecting production. I'd suggest the next steps are to bring up a borg-based backup server and put a few hosts into this. After running for a while, we can add all hosts, and then deprecate the current bup-based backup server in vexxhost and replace that with a borg-based one; giving us dual offsite backups. [1] https://borgbackup.readthedocs.io/en/stable/ Change-Id: I2a125f2fac11d8e3a3279eb7fa7adb33a3acaa4e
2020-07-16 13:43:18 +10:00
- playbooks/service-borg-backup.yaml
borg : match install-borg role to run deploy job This was forgotten in the original addition. Change-Id: I0725b99e938b993e68106f6ba1b2704e9413e902
2020-10-12 13:06:10 +11:00
- playbooks/roles/install-borg/
Add borg-backup roles This adds roles to implement backup with borg [1]. Our current tool "bup" has no Python 3 support and is not packaged for Ubuntu Focal. This means it is effectively end-of-life. borg fits our model of servers backing themselves up to a central location, is well documented and seems well supported. It also has the clarkb seal of approval :) As mentioned, borg works in the same manner as bup by doing an efficient back up over ssh to a remote server. The core of these roles are the same as the bup based ones; in terms of creating a separate user for each host and deploying keys and ssh config. This chooses to install borg in a virtualenv on /opt. This was chosen for a number of reasons; firstly reading the history of borg there have been incompatible updates (although they provide a tool to update repository formats); it seems important that we both pin the version we are using and keep clients and server in sync. Since we have a hetrogenous distribution collection we don't want to rely on the packaged tools which may differ. I don't feel like this is a great application for a container; we actually don't want it that isolated from the base system because it's goal is to read and copy it offsite with as little chance of things going wrong as possible. Borg has a lot of support for encrypting the data at rest in various ways. However, that introduces the possibility we could lose both the key and the backup data. Really the only thing stopping this is key management, and if we want to go down this path we can do it as a follow-on. The remote end server is configured via ssh command rules to run in append-only mode. This means a misbehaving client can't delete its old backups. In theory we can prune backups on the server side -- something we could not do with bup. The documentation has been updated but is vague on this part; I think we should get some hosts in operation, see how the de-duplication is working out and then decide how we want to mange things long term. Testing is added; a focal and bionic host both run a full backup of themselves to the backup server. Pretty cool, the logs are in /var/log/borg-backup-<host>.log. No hosts are currently in the borg groups, so this can be applied without affecting production. I'd suggest the next steps are to bring up a borg-based backup server and put a few hosts into this. After running for a while, we can add all hosts, and then deprecate the current bup-based backup server in vexxhost and replace that with a borg-based one; giving us dual offsite backups. [1] https://borgbackup.readthedocs.io/en/stable/ Change-Id: I2a125f2fac11d8e3a3279eb7fa7adb33a3acaa4e
2020-07-16 13:43:18 +10:00
- playbooks/roles/borg-backup/
- playbooks/roles/borg-backup-server/
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- job:
name: infra-prod-service-registry
parent: infra-prod-service-base
description: Run service-registry.yaml playbook.
vars:
playbook_name: service-registry.yaml
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/service-registry.yaml
Split inventory into multiple dirs and move hostvars Make inventory/service for service-specific things, including the groups.yaml group definitions, and inventory/base for hostvars related to the base system, including the list of hosts. Move the exisitng host_vars into inventory/service, since most of them are likely service-specific. Move group_vars/all.yaml into base/group_vars as almost all of it is related to base things, with the execption of the gerrit public key. A followup patch will move host-specific values into equivilent files in inventory/base. This should let us override hostvars in gate jobs. It should also allow us to do better file matchers - and to be able to organize our playbooks move if we want to. Depends-On: https://review.opendev.org/731583 Change-Id: Iddf57b5be47c2e9de16b83a1bc83bee25db995cf
2020-05-26 15:46:41 -05:00
- inventory/service/group_vars/registry.yaml
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/pip3/
- playbooks/roles/install-docker/
Run iptables in service playbooks instead of base It's the only part of base that's important to run when we run a service. Run it in the service playbooks and get rid of the dependency on infra-prod-base. Continue running it in base so that new nodes are brought up with iptables in place. Bump the timeout for the mirror job, because the iptables addition seems to have just bumped it over the edge. Change-Id: I4608216f7a59cfa96d3bdb191edd9bc7bb9cca39
2020-05-26 17:41:54 -05:00
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/registry/
- job:
name: infra-prod-service-zuul-preview
parent: infra-prod-service-base
description: Run service-zuul-preview.yaml playbook.
vars:
playbook_name: service-zuul-preview.yaml
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/service-zuul-preview.yaml
Split inventory into multiple dirs and move hostvars Make inventory/service for service-specific things, including the groups.yaml group definitions, and inventory/base for hostvars related to the base system, including the list of hosts. Move the exisitng host_vars into inventory/service, since most of them are likely service-specific. Move group_vars/all.yaml into base/group_vars as almost all of it is related to base things, with the execption of the gerrit public key. A followup patch will move host-specific values into equivilent files in inventory/base. This should let us override hostvars in gate jobs. It should also allow us to do better file matchers - and to be able to organize our playbooks move if we want to. Depends-On: https://review.opendev.org/731583 Change-Id: Iddf57b5be47c2e9de16b83a1bc83bee25db995cf
2020-05-26 15:46:41 -05:00
- inventory/service/group_vars/zuul-preview.yaml
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/pip3/
- playbooks/roles/install-docker/
Run iptables in service playbooks instead of base It's the only part of base that's important to run when we run a service. Run it in the service playbooks and get rid of the dependency on infra-prod-base. Continue running it in base so that new nodes are brought up with iptables in place. Bump the timeout for the mirror job, because the iptables addition seems to have just bumped it over the edge. Change-Id: I4608216f7a59cfa96d3bdb191edd9bc7bb9cca39
2020-05-26 17:41:54 -05:00
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/zuul-preview/
- job:
name: infra-prod-service-zookeeper
parent: infra-prod-service-base
description: Run service-zookeeper.yaml playbook.
vars:
playbook_name: service-zookeeper.yaml
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
Split inventory into multiple dirs and move hostvars Make inventory/service for service-specific things, including the groups.yaml group definitions, and inventory/base for hostvars related to the base system, including the list of hosts. Move the exisitng host_vars into inventory/service, since most of them are likely service-specific. Move group_vars/all.yaml into base/group_vars as almost all of it is related to base things, with the execption of the gerrit public key. A followup patch will move host-specific values into equivilent files in inventory/base. This should let us override hostvars in gate jobs. It should also allow us to do better file matchers - and to be able to organize our playbooks move if we want to. Depends-On: https://review.opendev.org/731583 Change-Id: Iddf57b5be47c2e9de16b83a1bc83bee25db995cf
2020-05-26 15:46:41 -05:00
- inventory/service/group_vars/zookeeper.yaml
- ^inventory/service/host_vars/zk\d+\..*
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/pip3/
- playbooks/roles/install-docker/
Run iptables in service playbooks instead of base It's the only part of base that's important to run when we run a service. Run it in the service playbooks and get rid of the dependency on infra-prod-base. Continue running it in base so that new nodes are brought up with iptables in place. Bump the timeout for the mirror job, because the iptables addition seems to have just bumped it over the edge. Change-Id: I4608216f7a59cfa96d3bdb191edd9bc7bb9cca39
2020-05-26 17:41:54 -05:00
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/zookeeper/
- job:
name: infra-prod-service-zuul
parent: infra-prod-service-base
description: |
Run service-zuul.yaml playbook.
This configures the main Zuul cluster. It will perform a
smart-reconfigure of the scheduler if the tenant configuration
is changed.
vars:
playbook_name: service-zuul.yaml
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/service-zuul.yaml
Split inventory into multiple dirs and move hostvars Make inventory/service for service-specific things, including the groups.yaml group definitions, and inventory/base for hostvars related to the base system, including the list of hosts. Move the exisitng host_vars into inventory/service, since most of them are likely service-specific. Move group_vars/all.yaml into base/group_vars as almost all of it is related to base things, with the execption of the gerrit public key. A followup patch will move host-specific values into equivilent files in inventory/base. This should let us override hostvars in gate jobs. It should also allow us to do better file matchers - and to be able to organize our playbooks move if we want to. Depends-On: https://review.opendev.org/731583 Change-Id: Iddf57b5be47c2e9de16b83a1bc83bee25db995cf
2020-05-26 15:46:41 -05:00
- inventory/service/group_vars/zuul
- inventory/service/group_vars/zookeeper.yaml
- inventory/service/host_vars/zk\d+
Run zuul-web on zuul01 and add to load balancer Change-Id: Ia8b10338fa3a1876993404276e0759f4b10d6b54
2022-03-04 12:31:47 -08:00
- inventory/service/host_vars/zuul\d+.opendev.org
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/install-docker/
Run iptables in service playbooks instead of base It's the only part of base that's important to run when we run a service. Run it in the service playbooks and get rid of the dependency on infra-prod-base. Continue running it in base so that new nodes are brought up with iptables in place. Bump the timeout for the mirror job, because the iptables addition seems to have just bumped it over the edge. Change-Id: I4608216f7a59cfa96d3bdb191edd9bc7bb9cca39
2020-05-26 17:41:54 -05:00
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/zookeeper/
- playbooks/roles/zuul
Fix up openafs-client job matching The jobs should have file matchers for "roles/openafs-client" (not playbooks/). Fix this. Add the openafs/kerberos role matchers to Zuul as well, as it uses them on the executors. Change-Id: I66fd7792d6b533362606291e1bfc01dfa2a2e05b
2021-03-03 13:41:56 +11:00
- roles/kerberos-client/
- roles/openafs-client/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
Add a standalone zuul db server Change-Id: Ibb260f820dbc1d9d6ca523ff3903134612cb003e
2024-04-04 09:07:19 -07:00
- job:
name: infra-prod-service-zuul-db
parent: infra-prod-service-base
description: Run service-zuul-db.yaml playbook.
vars:
playbook_name: service-zuul-db.yaml
files:
- inventory/base
- playbooks/service-zuul-db.yaml
- inventory/service/group_vars/zuul-db.yaml
- playbooks/roles/iptables/
- playbooks/roles/install-docker/
- playbooks/roles/mariadb/
Add Zuul load balancer This adds a load balancer for zuul-web and fingergw. Change-Id: Id5aa01151f64f3c85e1532ad66999ef9471c5896
2022-02-10 09:28:46 -08:00
- job:
name: infra-prod-service-zuul-lb
parent: infra-prod-service-base
description: Run service-zuul-lb.yaml playbook.
vars:
playbook_name: service-zuul-lb.yaml
files:
- inventory/base
- playbooks/service-zuul-lb.yaml
- inventory/service/group_vars/zuul-lb.yaml
- playbooks/roles/pip3/
- playbooks/roles/iptables/
- playbooks/roles/install-docker/
- playbooks/roles/haproxy/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- job:
name: infra-prod-service-review
parent: infra-prod-service-base
description: Run service-review.yaml playbook.
vars:
playbook_name: service-review.yaml
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/service-review.yaml
Replace testing group vars with host vars for review02 Previously we had a test specific group vars file for the review Ansible group. This provided junk secrets to our test installations of Gerrit then we relied on the review02.opendev.org production host vars file to set values that are public. Unfortunately, this meant we were using the production heapLimit value which is far too large for our test instances leading to the occasionaly failure: There is insufficient memory for the Java Runtime Environment to continue. Native memory allocation (mmap) failed to map 9596567552 bytes for committing reserved memory. We cannot set the heapLimit in the group var file because the hostvar file overrides those values. To fix this we need to replace the test specific group var contents with a test specific host var file instead. To avoid repeating ourselves we also create a new review.yaml group_vars file to capture common settings between testing and prod. Note we should look at combining this new file with the gerrit.yaml group_vars. On the testing side of things we set the heapLimit to 6GB, we change the serverid value to prevent any unexpected notedb confusion, and we remove replication config. Change-Id: Id8ec5cae967cc38acf79ecf18d3a0faac3a9c4b3
2021-10-11 13:22:50 -07:00
- inventory/service/group_vars/review.yaml
Fix infra-prod-service-review file matchers We changed review01.openstack.org to review02.openstack.org in the host var file matchers for this job thinking that was the issue previously. Unfortunately the actual file is review02.opendev.org. Update the matcher again to actually trigger the job. We also make a small edit to the gerrit role's README to ensure we trigger the job when this change lands. Change-Id: I1f235d0ddbb2d7f400ea2e99ffabdf5db35671a1
2023-03-03 11:47:02 -08:00
- inventory/service/host_vars/review02.opendev.org.yaml
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/pip3/
- playbooks/roles/install-docker/
Run iptables in service playbooks instead of base It's the only part of base that's important to run when we run a service. Run it in the service playbooks and get rid of the dependency on infra-prod-base. Continue running it in base so that new nodes are brought up with iptables in place. Bump the timeout for the mirror job, because the iptables addition seems to have just bumped it over the edge. Change-Id: I4608216f7a59cfa96d3bdb191edd9bc7bb9cca39
2020-05-26 17:41:54 -05:00
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/gerrit/
Trigger gerrit image promotion when the gerrit image jobs update We often need to update gerrit image build details that only live in the job specification. For example tag or branch versions of gerrit and related repos. When we do this if we don't also do a noop update to our Dockerfiles the promotion job doesn't run for these images because we can't do the implicit file match in the promotion (deploy) pipeline. Fix this by explicitly matching the job config file in our jobs so that when we update the gerrit jobs we also run the gerrit image promotion jobs. We also ensure the system-config-run-review and infra-prod-service-review jobs are triggered when the docker image jobs update. This ensures we actually test the resulting images and then perform potentially necessary deployment actions before they are pulled into use. Change-Id: Id0c51818cd1e01bd16a79ab0c0f9172e844376b8
2023-11-29 09:59:30 -08:00
- zuul.d/docker-images/gerrit.yaml
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
refstack: add production image and deployment jobs Change-Id: I017a32ee374f0473525c9941c41b26c2a43bf2c8
2021-02-09 16:15:08 +11:00
- job:
name: infra-prod-service-refstack
parent: infra-prod-service-base
description: Run service-refstack.yaml playbook.
vars:
playbook_name: service-refstack.yaml
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
refstack: add production image and deployment jobs Change-Id: I017a32ee374f0473525c9941c41b26c2a43bf2c8
2021-02-09 16:15:08 +11:00
- playbooks/service-refstack.yaml
Don't run infra-prod-run-refstack on all group var updates This was running on all group var updates but we only need to run it when refstack group vars update. Change the file requirements to match the refstack.yaml group file to address this. Change-Id: Id5ed4b65c1ed6566696fea9a33db27e9318af1a6
2022-03-04 15:30:47 -08:00
- inventory/service/group_vars/refstack.yaml
refstack: add production image and deployment jobs Change-Id: I017a32ee374f0473525c9941c41b26c2a43bf2c8
2021-02-09 16:15:08 +11:00
- inventory/service/host_vars/refstack[0-9][0-9]
- playbooks/roles/install-docker/
- playbooks/roles/pip3/
refstack: fix typo in role matcher Change-Id: I61929708be87a28669606ac38abf478afd70fc51
2021-02-11 10:37:31 +11:00
- playbooks/roles/refstack/
refstack: add production image and deployment jobs Change-Id: I017a32ee374f0473525c9941c41b26c2a43bf2c8
2021-02-09 16:15:08 +11:00
- playbooks/roles/iptables/
- playbooks/roles/logrotate/
- docker/refstack
- docker/python-base/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- job:
name: infra-prod-service-gitea
parent: infra-prod-service-base
description: Run service-gitea.yaml playbook.
vars:
playbook_name: service-gitea.yaml
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/service-gitea.yaml
Split inventory into multiple dirs and move hostvars Make inventory/service for service-specific things, including the groups.yaml group definitions, and inventory/base for hostvars related to the base system, including the list of hosts. Move the exisitng host_vars into inventory/service, since most of them are likely service-specific. Move group_vars/all.yaml into base/group_vars as almost all of it is related to base things, with the execption of the gerrit public key. A followup patch will move host-specific values into equivilent files in inventory/base. This should let us override hostvars in gate jobs. It should also allow us to do better file matchers - and to be able to organize our playbooks move if we want to. Depends-On: https://review.opendev.org/731583 Change-Id: Iddf57b5be47c2e9de16b83a1bc83bee25db995cf
2020-05-26 15:46:41 -05:00
- inventory/service/group_vars/gitea.yaml
- inventory/service/host_vars/gitea[0-9][0-9]
Run gitea and static tests when update Apache UA filters The gitea and static services both deploy apache using our UA filters. We were not testing these services when updating UA filters. This change fixes that giving us some basic sanity checking that UA filter updates are functional and quicker deployments of these filters. Change-Id: Icbe6558bb47946299a43905e2f64522576bad939
2023-05-02 10:16:29 -07:00
- playbooks/roles/apache-ua-filter/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/install-docker/
- playbooks/roles/pip3/
- playbooks/roles/gitea/
Run iptables in service playbooks instead of base It's the only part of base that's important to run when we run a service. Run it in the service playbooks and get rid of the dependency on infra-prod-base. Continue running it in base so that new nodes are brought up with iptables in place. Bump the timeout for the mirror job, because the iptables addition seems to have just bumped it over the edge. Change-Id: I4608216f7a59cfa96d3bdb191edd9bc7bb9cca39
2020-05-26 17:41:54 -05:00
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/logrotate/
- docker/gitea/
- docker/gitea-init/
- docker/jinja-init/
- docker/python-base/
- job:
name: infra-prod-service-eavesdrop
parent: infra-prod-service-base
description: Run service-eavesdrop.yaml playbook.
required-projects:
- opendev/system-config
- openstack/project-config
vars:
playbook_name: service-eavesdrop.yaml
files: &infra_prod_eavesdrop_files
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/service-eavesdrop.yaml
- playbooks/run-accessbot.yaml
Split inventory into multiple dirs and move hostvars Make inventory/service for service-specific things, including the groups.yaml group definitions, and inventory/base for hostvars related to the base system, including the list of hosts. Move the exisitng host_vars into inventory/service, since most of them are likely service-specific. Move group_vars/all.yaml into base/group_vars as almost all of it is related to base things, with the execption of the gerrit public key. A followup patch will move host-specific values into equivilent files in inventory/base. This should let us override hostvars in gate jobs. It should also allow us to do better file matchers - and to be able to organize our playbooks move if we want to. Depends-On: https://review.opendev.org/731583 Change-Id: Iddf57b5be47c2e9de16b83a1bc83bee25db995cf
2020-05-26 15:46:41 -05:00
- inventory/service/group_vars/eavesdrop.yaml
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/install-docker
Run iptables in service playbooks instead of base It's the only part of base that's important to run when we run a service. Run it in the service playbooks and get rid of the dependency on infra-prod-base. Continue running it in base so that new nodes are brought up with iptables in place. Bump the timeout for the mirror job, because the iptables addition seems to have just bumped it over the edge. Change-Id: I4608216f7a59cfa96d3bdb191edd9bc7bb9cca39
2020-05-26 17:41:54 -05:00
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/accessbot
Update eavesdrop deploy job This was missed when adding the statusbot/ircbot containers Change-Id: I198da471b8a0dd648a8e9f1bfe41988561a745f8
2021-06-11 23:23:20 +10:00
- playbooks/roles/limnoria
Setting Up Ansible For ptgbot Heavily taken from statusbot, but removed wiki and twitter defaults. Change-Id: I7b1958dbe37e5d25b8fde746235c88a4d6763ffd
2021-08-02 03:43:25 -07:00
- playbooks/roles/ptgbot
Update eavesdrop deploy job This was missed when adding the statusbot/ircbot containers Change-Id: I198da471b8a0dd648a8e9f1bfe41988561a745f8
2021-06-11 23:23:20 +10:00
- playbooks/roles/statusbot
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/logrotate
Run matrix-eavesdrop on eavesdrop Thin runs the new matrix-eavesdrop bot on the eavesdrop server. It will write logs out to the limnoria logs directory, which is mounted inside the container. Change-Id: I867eec692f63099b295a37a028ee096c24109a2e
2021-07-09 18:39:59 -07:00
- playbooks/roles/matrix-eavesdrop
Run matrix-gerritbot on eavesdrop Thin runs the new gerritbot-matrix bot on the eavesdrop server. Change-Id: Ic11ca46aa4da61d5b80a8996ad900fdf83ab70dc
2021-07-12 13:54:18 +00:00
- playbooks/roles/matrix-gerritbot
Run matrix-eavesdrop on eavesdrop Thin runs the new matrix-eavesdrop bot on the eavesdrop server. It will write logs out to the limnoria logs directory, which is mounted inside the container. Change-Id: I867eec692f63099b295a37a028ee096c24109a2e
2021-07-09 18:39:59 -07:00
- playbooks/zuul/templates/group_vars/eavesdrop.yaml.j2
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- docker/accessbot/
Update eavesdrop deploy job This was missed when adding the statusbot/ircbot containers Change-Id: I198da471b8a0dd648a8e9f1bfe41988561a745f8
2021-06-11 23:23:20 +10:00
- docker/ircbot
Run matrix-eavesdrop on eavesdrop Thin runs the new matrix-eavesdrop bot on the eavesdrop server. It will write logs out to the limnoria logs directory, which is mounted inside the container. Change-Id: I867eec692f63099b295a37a028ee096c24109a2e
2021-07-09 18:39:59 -07:00
- docker/matrix-eavesdrop
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- job:
name: infra-prod-run-accessbot
parent: infra-prod-service-base
description: Run run-accessbot.yaml playbook.
required-projects:
- opendev/system-config
- openstack/project-config
vars:
playbook_name: run-accessbot.yaml
Run accessbot less frequently We already run accessbot in project-config when the accessbot script changes. We don't need to run it whenever any of the puppet or other config on eavesdrop runs, not do we need to run it hourly. Just run it nightly and on changes to the actual accessbot config. Change-Id: Idd47f7c96f677fd1e1c8da3be262a52a70646acd
2020-05-08 08:12:37 -05:00
files:
- accessbot/channels.yaml
- playbooks/run-accessbot.yaml
- playbooks/roles/accessbot
- docker/accessbot/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
Migrate codesearch site to container The hound project has undergone a small re-birth and moved to https://github.com/hound-search/hound which has broken our deployment. We've talked about leaving codesearch up to gitea, but it's not quite there yet. There seems to be no point working on the puppet now. This builds a container than runs houndd. It's an opendev specific container; the config is pulled from project-config directly. There's some custom scripts that drive things. Some points for reviewers: - update-hound-config.sh uses "create-hound-config" (which is in jeepyb for historical reasons) to generate the config file. It grabs the latest projects.yaml from project-config and exits with a return code to indicate if things changed. - when the container starts, it runs update-hound-config.sh to populate the initial config. There is a testing environment flag and small config so it doesn't have to clone the entire opendev for functional testing. - it runs under supervisord so we can restart the daemon when projects are updated. Unlike earlier versions that didn't start listening till indexing was done, this version now puts up a "Hound is not ready yet" message when while it is working; so we can drop all the magic we were doing to probe if hound is listening via netstat and making Apache redirect to a status page. - resync-hound.sh is run from an external cron job daily, and does this update and restart check. Since it only reloads if changes are made, this should be relatively rare anyway. - There is a PR to monitor the config file (https://github.com/hound-search/hound/pull/357) which would mean the restart is unnecessary. This would be good in the near and we could remove the cron job. - playbooks/roles/codesearch is unexciting and deploys the container, certificates and an apache proxy back to localhost:6080 where hound is listening. I've combined removal of the old puppet bits here as the "-codesearch" namespace was already being used. Change-Id: I8c773b5ea6b87e8f7dfd8db2556626f7b2500473
2020-11-17 17:13:46 +11:00
- job:
name: infra-prod-service-codesearch
parent: infra-prod-service-base
description: Run service-codesearch.yaml playbook.
vars:
playbook_name: service-codesearch.yaml
files:
- docker/hound/
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
Migrate codesearch site to container The hound project has undergone a small re-birth and moved to https://github.com/hound-search/hound which has broken our deployment. We've talked about leaving codesearch up to gitea, but it's not quite there yet. There seems to be no point working on the puppet now. This builds a container than runs houndd. It's an opendev specific container; the config is pulled from project-config directly. There's some custom scripts that drive things. Some points for reviewers: - update-hound-config.sh uses "create-hound-config" (which is in jeepyb for historical reasons) to generate the config file. It grabs the latest projects.yaml from project-config and exits with a return code to indicate if things changed. - when the container starts, it runs update-hound-config.sh to populate the initial config. There is a testing environment flag and small config so it doesn't have to clone the entire opendev for functional testing. - it runs under supervisord so we can restart the daemon when projects are updated. Unlike earlier versions that didn't start listening till indexing was done, this version now puts up a "Hound is not ready yet" message when while it is working; so we can drop all the magic we were doing to probe if hound is listening via netstat and making Apache redirect to a status page. - resync-hound.sh is run from an external cron job daily, and does this update and restart check. Since it only reloads if changes are made, this should be relatively rare anyway. - There is a PR to monitor the config file (https://github.com/hound-search/hound/pull/357) which would mean the restart is unnecessary. This would be good in the near and we could remove the cron job. - playbooks/roles/codesearch is unexciting and deploys the container, certificates and an apache proxy back to localhost:6080 where hound is listening. I've combined removal of the old puppet bits here as the "-codesearch" namespace was already being used. Change-Id: I8c773b5ea6b87e8f7dfd8db2556626f7b2500473
2020-11-17 17:13:46 +11:00
- playbooks/service-codesearch.yaml
- inventory/service/host_vars/codesearch01.opendev.yaml
- inventory/service/group_vars/codesearch
- playbooks/roles/install-docker/
- playbooks/roles/pip3/
- playbooks/roles/codesearch
- playbooks/roles/logrotate
- playbooks/roles/iptables
Grafana container deployment This uses the Grafana container created with Iddfafe852166fe95b3e433420e2e2a4a6380fc64 to run the grafana.opendev.org service. We retain the old model of an Apache reverse-proxy; it's well tested and understood, it's much easier than trying to map all the SSL termination/renewal/etc. into the Grafana container and we don't have to convince ourselves the container is safe to be directly web-facing. Otherwise this is a fairly straight forward deployment of the container. As before, it uses the graph configuration kept in project-config which is loaded in with grafyaml, which is included in the container. Once nice advantage is that it makes it quite easy to develop graphs locally, using the container which can talk to the public graphite instance. The documentation has been updated with a reference on how to do this. Change-Id: I0cc76d29b6911aecfebc71e5fdfe7cf4fcd071a4
2020-06-23 11:41:27 +10:00
- job:
name: infra-prod-service-grafana
parent: infra-prod-service-base
description: Run service-grafana.yaml playbook.
vars:
playbook_name: service-grafana.yaml
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
Grafana container deployment This uses the Grafana container created with Iddfafe852166fe95b3e433420e2e2a4a6380fc64 to run the grafana.opendev.org service. We retain the old model of an Apache reverse-proxy; it's well tested and understood, it's much easier than trying to map all the SSL termination/renewal/etc. into the Grafana container and we don't have to convince ourselves the container is safe to be directly web-facing. Otherwise this is a fairly straight forward deployment of the container. As before, it uses the graph configuration kept in project-config which is loaded in with grafyaml, which is included in the container. Once nice advantage is that it makes it quite easy to develop graphs locally, using the container which can talk to the public graphite instance. The documentation has been updated with a reference on how to do this. Change-Id: I0cc76d29b6911aecfebc71e5fdfe7cf4fcd071a4
2020-06-23 11:41:27 +10:00
- playbooks/service-grafana.yaml
- inventory/service/host_vars/grafana01.org.yaml
- inventory/service/group_vars/grafana
- playbooks/roles/install-docker/
- playbooks/roles/pip3/
- playbooks/roles/grafana
- playbooks/roles/logrotate
- playbooks/roles/iptables/
Graphite container deployment This deploys graphite from the upstream container. We override the statsd configuration to have it listen on ipv6. Similarly we override the ngnix config to listen on ipv6, enable ssl, forward port 80 to 443, block the /admin page (we don't use it). For production we will just want to put some cinder storage in /opt/graphite/storage on the production host and figure out how to migrate the old stats. The is also a bit of cleanup that will follow, because we half-converted grafana01.opendev.org -- so everything can't be in the same group till that is gone. Testing has been added to push some stats and ensure they are seen. Change-Id: Ie843b3d90a72564ef90805f820c8abc61a71017d
2020-06-26 10:23:16 +10:00
- job:
name: infra-prod-service-graphite
parent: infra-prod-service-base
description: Run service-graphite.yaml playbook.
vars:
playbook_name: service-graphite.yaml
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
Graphite container deployment This deploys graphite from the upstream container. We override the statsd configuration to have it listen on ipv6. Similarly we override the ngnix config to listen on ipv6, enable ssl, forward port 80 to 443, block the /admin page (we don't use it). For production we will just want to put some cinder storage in /opt/graphite/storage on the production host and figure out how to migrate the old stats. The is also a bit of cleanup that will follow, because we half-converted grafana01.opendev.org -- so everything can't be in the same group till that is gone. Testing has been added to push some stats and ensure they are seen. Change-Id: Ie843b3d90a72564ef90805f820c8abc61a71017d
2020-06-26 10:23:16 +10:00
- playbooks/service-graphite.yaml
- inventory/service/host_vars/graphite02.opendev.org.yaml
- inventory/service/group_vars/graphite
- playbooks/roles/install-docker/
- playbooks/roles/pip3/
- playbooks/roles/graphite/
- playbooks/roles/iptables/
Add a mailman3 list server This should now be a largely functional deployment of mailman 3. There are still some bits that need testing but we'll use followup changes to force failure and hold nodes. This deployment of mailman3 uses upstream docker container images. We currently hack up uids and gids to accomodate that. We also hack up the settings file and bind mount it over the upstream file in order to use host networking. We override the hyperkitty index type to xapian. All list domains are hosted in a single installation and we use native vhosting to handle that. We'll deploy this to a new server and migrate one mailing list domain at a time. This will allow us to start with lists.opendev.org and test things like dmarc settings before expanding to the remaining lists. A migration script is also included, which has seen extensive testing on held nodes for importing copies of the production data sets. Change-Id: Ic9bf5cfaf0b87c100a6ce003a6645010a7b50358
2022-07-27 11:35:41 -07:00
- job:
name: infra-prod-service-lists3
parent: infra-prod-service-base
description: Run service-lists3.yaml playbook.
vars:
playbook_name: service-lists3.yaml
files:
Trigger mm3 deployment when containers change Add the docker/mailman tree to the infra-prod-service-lists3 job so that we deploy new versions whenever we make changes to the container images. Change-Id: Ife5e878b1f81c2879c2959fe6d4de22fe841583b
2023-08-25 16:33:34 +00:00
- docker/mailman
Add a mailman3 list server This should now be a largely functional deployment of mailman 3. There are still some bits that need testing but we'll use followup changes to force failure and hold nodes. This deployment of mailman3 uses upstream docker container images. We currently hack up uids and gids to accomodate that. We also hack up the settings file and bind mount it over the upstream file in order to use host networking. We override the hyperkitty index type to xapian. All list domains are hosted in a single installation and we use native vhosting to handle that. We'll deploy this to a new server and migrate one mailing list domain at a time. This will allow us to start with lists.opendev.org and test things like dmarc settings before expanding to the remaining lists. A migration script is also included, which has seen extensive testing on held nodes for importing copies of the production data sets. Change-Id: Ic9bf5cfaf0b87c100a6ce003a6645010a7b50358
2022-07-27 11:35:41 -07:00
- inventory/base
Fix the relevant files lists for lists3 jobs Fix the infra-prod-service-lists3 job to trigger when we update the mailman3.yaml group vars file. In addition we make a noop reorganization change to the mailman3 group file to group exim vars together which will be used to ensure that this change triggers the lists3 job as expected. In system-config-run-lists3 we update that job to be triggered when we update the docker images for mailman. We don't bother testing this now as that would be masked off by the update to the mailman3 groups file. But in the future when we do mailman3 image updates we'll be looking for this job to run. Change-Id: I994b0a79bf46f525dd9e059719f5a08c9c390b8c
2023-10-15 19:52:01 -07:00
- inventory/service/group_vars/mailman3.yaml
Add a mailman3 list server This should now be a largely functional deployment of mailman 3. There are still some bits that need testing but we'll use followup changes to force failure and hold nodes. This deployment of mailman3 uses upstream docker container images. We currently hack up uids and gids to accomodate that. We also hack up the settings file and bind mount it over the upstream file in order to use host networking. We override the hyperkitty index type to xapian. All list domains are hosted in a single installation and we use native vhosting to handle that. We'll deploy this to a new server and migrate one mailing list domain at a time. This will allow us to start with lists.opendev.org and test things like dmarc settings before expanding to the remaining lists. A migration script is also included, which has seen extensive testing on held nodes for importing copies of the production data sets. Change-Id: Ic9bf5cfaf0b87c100a6ce003a6645010a7b50358
2022-07-27 11:35:41 -07:00
- playbooks/roles/iptables/
- playbooks/roles/base/exim
- playbooks/roles/mailman3/
- playbooks/service-lists3.yaml
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
# Run AFS changes separately so we can make sure to only do one at a time
# (turns out quorum is nice to have)
- job:
openafs-server : add ansible roles for OpenAFS servers This starts at migrating OpenAFS server setup to Ansible. Firstly we split up the groups and explicitly name hosts, as we will me migrating each one step-by-step. We split out 1.8 hosts into a new afs-1.8 group; the first host is afs01.ord.openstack.org which already has openafs 1.8 installed manually. An openafs-server role is introduced that does the same setup as the extant puppet. The AFS job is renamed to infra-prod-afs as the puppet component will eventually disappear. Otherwise it runs in the same way, but also runs the openafs-server role for the 1.8 servers. Once this is merged, we can run it against afs01.ord.openstack.org to ensure it works and is idempotent. We can then take on upgrading the other file servers, and work further on the database servers. Change-Id: I7998af43961999412f58a78214f4b5387713d30e
2021-01-18 15:30:35 +11:00
name: infra-prod-service-afs
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
parent: infra-prod-service-base
openafs-server : add ansible roles for OpenAFS servers This starts at migrating OpenAFS server setup to Ansible. Firstly we split up the groups and explicitly name hosts, as we will me migrating each one step-by-step. We split out 1.8 hosts into a new afs-1.8 group; the first host is afs01.ord.openstack.org which already has openafs 1.8 installed manually. An openafs-server role is introduced that does the same setup as the extant puppet. The AFS job is renamed to infra-prod-afs as the puppet component will eventually disappear. Otherwise it runs in the same way, but also runs the openafs-server role for the 1.8 servers. Once this is merged, we can run it against afs01.ord.openstack.org to ensure it works and is idempotent. We can then take on upgrading the other file servers, and work further on the database servers. Change-Id: I7998af43961999412f58a78214f4b5387713d30e
2021-01-18 15:30:35 +11:00
description: Run AFS playbook.
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
vars:
openafs-server : add ansible roles for OpenAFS servers This starts at migrating OpenAFS server setup to Ansible. Firstly we split up the groups and explicitly name hosts, as we will me migrating each one step-by-step. We split out 1.8 hosts into a new afs-1.8 group; the first host is afs01.ord.openstack.org which already has openafs 1.8 installed manually. An openafs-server role is introduced that does the same setup as the extant puppet. The AFS job is renamed to infra-prod-afs as the puppet component will eventually disappear. Otherwise it runs in the same way, but also runs the openafs-server role for the 1.8 servers. Once this is merged, we can run it against afs01.ord.openstack.org to ensure it works and is idempotent. We can then take on upgrading the other file servers, and work further on the database servers. Change-Id: I7998af43961999412f58a78214f4b5387713d30e
2021-01-18 15:30:35 +11:00
playbook_name: service-afs.yaml
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
infra_prod_ansible_forks: 1
required-projects:
- opendev/system-config
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
openafs-server : add ansible roles for OpenAFS servers This starts at migrating OpenAFS server setup to Ansible. Firstly we split up the groups and explicitly name hosts, as we will me migrating each one step-by-step. We split out 1.8 hosts into a new afs-1.8 group; the first host is afs01.ord.openstack.org which already has openafs 1.8 installed manually. An openafs-server role is introduced that does the same setup as the extant puppet. The AFS job is renamed to infra-prod-afs as the puppet component will eventually disappear. Otherwise it runs in the same way, but also runs the openafs-server role for the 1.8 servers. Once this is merged, we can run it against afs01.ord.openstack.org to ensure it works and is idempotent. We can then take on upgrading the other file servers, and work further on the database servers. Change-Id: I7998af43961999412f58a78214f4b5387713d30e
2021-01-18 15:30:35 +11:00
- playbooks/service-afs.yaml
Split inventory into multiple dirs and move hostvars Make inventory/service for service-specific things, including the groups.yaml group definitions, and inventory/base for hostvars related to the base system, including the list of hosts. Move the exisitng host_vars into inventory/service, since most of them are likely service-specific. Move group_vars/all.yaml into base/group_vars as almost all of it is related to base things, with the execption of the gerrit public key. A followup patch will move host-specific values into equivilent files in inventory/base. This should let us override hostvars in gate jobs. It should also allow us to do better file matchers - and to be able to organize our playbooks move if we want to. Depends-On: https://review.opendev.org/731583 Change-Id: Iddf57b5be47c2e9de16b83a1bc83bee25db995cf
2020-05-26 15:46:41 -05:00
- inventory/service/group_vars/afs
- inventory/service/group_vars/mirror-update
Run iptables in service playbooks instead of base It's the only part of base that's important to run when we run a service. Run it in the service playbooks and get rid of the dependency on infra-prod-base. Continue running it in base so that new nodes are brought up with iptables in place. Bump the timeout for the mirror job, because the iptables addition seems to have just bumped it over the edge. Change-Id: I4608216f7a59cfa96d3bdb191edd9bc7bb9cca39
2020-05-26 17:41:54 -05:00
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/vos-release/
openafs-server : add ansible roles for OpenAFS servers This starts at migrating OpenAFS server setup to Ansible. Firstly we split up the groups and explicitly name hosts, as we will me migrating each one step-by-step. We split out 1.8 hosts into a new afs-1.8 group; the first host is afs01.ord.openstack.org which already has openafs 1.8 installed manually. An openafs-server role is introduced that does the same setup as the extant puppet. The AFS job is renamed to infra-prod-afs as the puppet component will eventually disappear. Otherwise it runs in the same way, but also runs the openafs-server role for the 1.8 servers. Once this is merged, we can run it against afs01.ord.openstack.org to ensure it works and is idempotent. We can then take on upgrading the other file servers, and work further on the database servers. Change-Id: I7998af43961999412f58a78214f4b5387713d30e
2021-01-18 15:30:35 +11:00
- playbooks/roles/openafs-server/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- modules/
- manifests/
Fix up openafs-client job matching The jobs should have file matchers for "roles/openafs-client" (not playbooks/). Fix this. Add the openafs/kerberos role matchers to Zuul as well, as it uses them on the executors. Change-Id: I66fd7792d6b533362606291e1bfc01dfa2a2e05b
2021-03-03 13:41:56 +11:00
- roles/kerberos-client/
- roles/openafs-client/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
kerberos-kdc: role to manage Kerberos KDC servers This adds a role and related testing to manage our Kerberos KDC servers, intended to replace the puppet modules currently performing this task. This role automates realm creation, initial setup, key material distribution and replica host configuration. None of this is intended to run on the production servers which are already setup with an active database, and the role should be effectively idempotent in production. Note that this does not yet switch the production servers into the new groups; this can be done in a separate step under controlled conditions and with related upgrades of the host OS to Focal. Change-Id: I60b40897486b29beafc76025790c501b5055313d
2021-03-05 16:10:01 +11:00
- job:
name: infra-prod-service-kerberos
parent: infra-prod-service-base
description: Run Kerberos playbook.
vars:
playbook_name: service-kerberos.yaml
infra_prod_ansible_forks: 1
required-projects:
- opendev/system-config
files:
Restrict generic inventory matchers to inventory/base We have a subdir in inventory called base that includes the shared files that we don't have a good way to distinguish between services. Limit the file matchers to inventory/base so that we don't trigger all of the services anytime a single service's host_vars changes. Change-Id: I3f461b4ab56ec55beca29e123186b36513803a44
2021-08-21 12:12:33 -05:00
- inventory/base
kerberos-kdc: role to manage Kerberos KDC servers This adds a role and related testing to manage our Kerberos KDC servers, intended to replace the puppet modules currently performing this task. This role automates realm creation, initial setup, key material distribution and replica host configuration. None of this is intended to run on the production servers which are already setup with an active database, and the role should be effectively idempotent in production. Note that this does not yet switch the production servers into the new groups; this can be done in a separate step under controlled conditions and with related upgrades of the host OS to Focal. Change-Id: I60b40897486b29beafc76025790c501b5055313d
2021-03-05 16:10:01 +11:00
- playbooks/service-kerberos.yaml
- inventory/service/group_vars/kerberos-kdc.yaml
- playbooks/roles/kerberos-kdc/
- roles/kerberos-client/
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- job:
name: infra-prod-remote-puppet-else
parent: infra-prod-service-base
description: Run remote-puppet-else.yaml playbook.
vars:
playbook_name: remote_puppet_else.yaml
infra_prod_ansible_forks: 50
required-projects:
- opendev/ansible-role-puppet
- opendev/system-config
files:
Run remote-puppet-else daily instead of hourly Update the file matchers to actually match the current set of puppet things. This ensure the deploy job runs when we want it and we can catch up daily instead of hourly. Previously a number of the matchers didn't actually match the puppet things because the path prefix was wrong or works were in different orders for the dir names. Change-Id: I3510da81d942cf6fb7da998b8a73b0a566ea7411
2021-08-17 15:54:38 -07:00
- Gemfile
- Rakefile
- modules.env
- install_modules.sh
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- hiera/
- inventory/
Run remote-puppet-else daily instead of hourly Update the file matchers to actually match the current set of puppet things. This ensure the deploy job runs when we want it and we can catch up daily instead of hourly. Previously a number of the matchers didn't actually match the puppet things because the path prefix was wrong or works were in different orders for the dir names. Change-Id: I3510da81d942cf6fb7da998b8a73b0a566ea7411
2021-08-17 15:54:38 -07:00
- roles/puppet-install/
- playbooks/install_puppet.yaml
- playbooks/update_puppet_version.yaml
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/remote_puppet_else.yaml
Run remote-puppet-else daily instead of hourly Update the file matchers to actually match the current set of puppet things. This ensure the deploy job runs when we want it and we can catch up daily instead of hourly. Previously a number of the matchers didn't actually match the puppet things because the path prefix was wrong or works were in different orders for the dir names. Change-Id: I3510da81d942cf6fb7da998b8a73b0a566ea7411
2021-08-17 15:54:38 -07:00
- playbooks/roles/puppet-run/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- playbooks/roles/install-ansible-roles/
- playbooks/roles/disable-puppet-agent/
Run remote-puppet-else daily instead of hourly Update the file matchers to actually match the current set of puppet things. This ensure the deploy job runs when we want it and we can catch up daily instead of hourly. Previously a number of the matchers didn't actually match the puppet things because the path prefix was wrong or works were in different orders for the dir names. Change-Id: I3510da81d942cf6fb7da998b8a73b0a566ea7411
2021-08-17 15:54:38 -07:00
- playbooks/roles/puppet-setup-ansible/
Run iptables in service playbooks instead of base It's the only part of base that's important to run when we run a service. Run it in the service playbooks and get rid of the dependency on infra-prod-base. Continue running it in base so that new nodes are brought up with iptables in place. Bump the timeout for the mirror job, because the iptables addition seems to have just bumped it over the edge. Change-Id: I4608216f7a59cfa96d3bdb191edd9bc7bb9cca39
2020-05-26 17:41:54 -05:00
- playbooks/roles/iptables/
Organize zuul jobs in zuul.d/ dir Our .zuul.yaml file has grown quite large. Try to make this more manageable by splitting it into zuul.d/ directory with jobs organized by function. Change-Id: I0739eb1e2bc64dcacebf92e25503f67302f7c882
2020-05-07 12:37:26 -07:00
- modules/
- manifests/
- job:
name: infra-prod-run-cloud-launcher
parent: infra-prod-service-base
description: Run cloud launcher playbook
vars:
playbook_name: run_cloud_launcher.yaml
infra_prod_ansible_forks: 1
required-projects:
- opendev/ansible-role-cloud-launcher
- opendev/system-config
files:
- playbooks/run_cloud_launcher.yaml
Abstract name of bastion host for testing path This replaces hard-coding of the host "bridge.openstack.org" with hard-coding of the first (and only) host in the group "bastion". The idea here is that we can, as much as possible, simply switch one place to an alternative hostname for the bastion such as "bridge.opendev.org" when we upgrade. This is just the testing path, for now; a follow-on will modify the production path (which doesn't really get speculatively tested) This needs to be defined in two places : 1) We need to define this in the run jobs for Zuul to use in the playbooks/zuul/run-*.yaml playbooks, as it sets up and collects logs from the testing bastion host. 2) The nested Ansible run will then use inventory inventory/service/groups.yaml Various other places are updated to use this abstracted group as the bastion host. Variables are moved into the bastion group (which only has one host -- the actual bastion host) which means we only have to update the group mapping to the new host. This is intended to be a no-op change; all the jobs should work the same, but just using the new abstractions. Change-Id: Iffb462371939989b03e5d6ac6c5df63aa7708513
2022-09-20 11:57:22 +10:00
- inventory/service/group_vars/bastion.yaml
infra-prod: run job against linaro We have access to manage the linaro cloud, but we don't want to completely own the host as it has been configured with kolla-ansible; so we don't want to take over things like name resolution, iptables rules, docker installation, etc. But we would like to manage some parts of it, like rolling out our root users, some cron jobs, etc. While we could just log in and do these things, it doesn't feel very openinfra. This allows us to have a group "unmanaged" that skips the base jobs. The base playbook is updated to skip these hosts. For now, we add a cloud-linaro prod job that just does nothing so we can validate the whole thing. When it's working, I plan to add a few things as discussed above. Change-Id: Ie8de70cbac7ffb9d727a06a349c3d2a3b3aa0b40
2023-03-15 12:00:25 +11:00
- job:
name: infra-prod-cloud-linaro
parent: infra-prod-service-base
description: Run management tasks against Linaro
vars:
playbook_name: service-cloud-linaro.yaml
required-projects:
- opendev/system-config
files:
- playbooks/service-cloud-linaro.yaml
Copy Permalink