236 Commits

Author SHA1 Message Date
Zuul
fe7d38af63 Merge "Remove system-config-puppet-beaker-rspec-puppet-4-centos-7-infra" 2020-04-21 18:09:45 +00:00
Zuul
fbcc45401b Merge "Use real passwords for meetpad" 2020-04-20 18:29:43 +00:00
Zuul
5c1c0d4a56 Merge "Move puppet apply jobs to system-config repo" 2020-04-20 17:30:56 +00:00
Zuul
f5cb448351 Merge "Remove puppet-beaker-rspec-puppet-4-infra-system-config" 2020-04-20 17:13:33 +00:00
Andreas Jaeger
ef2a21cc4d Remove system-config-puppet-beaker-rspec-puppet-4-centos-7-infra
This job is not used, remove it. OpenDev doesn't run CentOS 7 anymore.

Change-Id: I6865f24a6670d0c545a1e1014f7dd954d324b00d
2020-04-20 19:05:37 +02:00
James E. Blair
f7bf07a03d Use real passwords for meetpad
The docker containers expect this now and refuse to start with
fake passwords.

Change-Id: I4c4bd243c9684e3987eeb99e4c66d31a882336a0
2020-04-20 09:05:51 -07:00
Monty Taylor
fedd6c42f8 Fix remote_puppet playbook names
They have underscores, not dashes.

Change-Id: I6e3e0e96e38f24484e6038e502dfcc279e8e4fbf
2020-04-20 07:44:52 -05:00
Monty Taylor
ca684c775b Move puppet apply jobs to system-config repo
These use legacy-base, which sucks, but what sucks even more is
that they are in openstack-zuul-jobs, which makes them extra
awkward to try to adjust.

Change-Id: I87b3d56de41f0ba5658c1240ddfc7ecf1c3c43af
2020-04-19 10:57:22 -05:00
Monty Taylor
1a39133661 Remove puppet-beaker-rspec-puppet-4-infra-system-config
This doesn't actually do anything useful any more, but it spends
a lot of time not doing it.

Basically, this is only testing the things in
modules/openstack_project/spec/acceptance/basic_spec.rb, which
are things we install and test in ansible now.

There are related jobs, puppet-beaker-rspec-puppet-4-infra,
which are run on puppet- repos and run their rspec tests, but
that won't be affected by this.

Change-Id: I21b01d360b50dba10673c2986e8a2868b8747522
2020-04-19 10:57:22 -05:00
Zuul
faccef4594 Merge "Add install-docker and pip3 to files triggers" 2020-04-17 22:19:46 +00:00
Monty Taylor
7f5faf8eec Add install-docker and pip3 to files triggers
The jobs which use install-docker and pip3 should be triggered
by changes to install-docker or pip3.

Change-Id: Ia6ec8da72fee38377760cb27dd7df26fa169760b
2020-04-17 20:31:33 +00:00
Monty Taylor
f378793ac5 Remove semaphore from service-bridge
This is not intended to have its own semaphore.

Change-Id: I587b22f099c6031177e590dbb8dabd8c2ee58295
2020-04-17 15:29:06 -05:00
Zuul
b0ab2f37c5 Merge "Run ZK from containers" 2020-04-17 16:49:34 +00:00
Clark Boylan
e7878b1ddb Simplify .zuul.yaml regexes
Zuul uses an re.match() check on file list regexes. This means that the
leading ^ is redundant as is a trailing .*

Attempt to make this more clear by dropping those leading and trailing
regex operators to be consistent across the file. This makes the rules
easier to read and should make them easier to reason about.

Change-Id: Id4cd17d816c9af023a655bdadeedb9421e51cdca
2020-04-17 08:54:27 -07:00
James E. Blair
42574b2b37 Run ZK from containers
Migration plan:
* add zk* to emergency
* copy data files on each node to a safe place for DR backup
* make a json data backup: zk-shell localhost:2181 --run-once 'mirror / json://!tmp!zookeeper-backup.json/'
* manually run a modified playbook to set up the docker infra without starting containers
* rolling restart; for each node:
  * stop zk
  * split data and log files and move them to new locations
  * remove zk packages
  * start zk containers
* remove from emergency; land this change.

Change-Id: Ic06c9cf9604402aa8eb4bb79238021c14c5d9563
2020-04-17 08:43:09 -07:00
Zuul
e3ad9e79eb Merge "Get rid of all-clouds.yaml" 2020-04-16 15:41:55 +00:00
Zuul
af9f9898ae Merge "Remove infra-prod-update-system-config from etherpad" 2020-04-15 19:16:59 +00:00
Monty Taylor
6f5af9571e Remove infra-prod-update-system-config from etherpad
We missed a place removing the old system-config job.

Change-Id: Ie030435eb80b127cc3e1080bed0c258474e335ed
2020-04-15 13:18:07 -05:00
Monty Taylor
ebae022d07 Use project-config from zuul instead of direct clones
We use project-config for gerrit, gitea and nodepool config. That's
cool, because can clone that from zuul too and make sure that each
prod run we're doing runs with the contents of the patch in question.

Introduce a flag file that can be touched in /home/zuulcd that will
block zuul from running prod playbooks. By default, if the file is
there, zuul will wait for an hour before giving up.

Rename zuulcd to zuul

To better align prod and test, name the zuul user zuul.

Change-Id: I83c38c9c430218059579f3763e02d6b9f40c7b89
2020-04-15 12:29:33 -05:00
Monty Taylor
cb58a10d5b Run playbooks out of zuul checkout
Instead of running from system-config, run from the zuul prepared
git repo state. We already have a mutex of one, so we'll never
be fighting. This lets us land stacks of changes and be sure they
will accurately always use the correct git state.

As a todo, we should update manage-projects to do the same with
project-config.

Change-Id: I358554e344e12561d1f3063e0724f6b61d1f15a7
2020-04-14 14:54:23 -05:00
Monty Taylor
c117c1106d Update install-ansible away from /opt/system-config
So that we can start running things from the zuul source rather
thatn update-system-config and /opt/system-config, we need to
install a few things onto the host in install-ansible so that the
ansible env is standalone.

This introduces a split execution path. The ansible config is
now all installed globally onto the machine by install-ansible
and does not reference a git checkout.

For running ad-hoc commands, an ansible.cfg is introduced inside
the root of the system-config dir. So if ansible-playbook is
executed with PWD==/opt/system-config it will find that ansible.cfg,
it will take precedence, and any content from system-config
will take precedence.

As a followup we'll make /opt/system-config/ansible.cfg written
out by install-ansible from the same template, and we'll update
the split to make ansible only work when executed from one of
the two configured locations, so that it's clear where we're
operating from.

Change-Id: I097694244e95751d96e67304aaae53ad19d8b873
2020-04-14 14:54:23 -05:00
Zuul
4228eb91ea Merge "Publish docs updates to docs.opendev.org" 2020-04-14 19:51:56 +00:00
Monty Taylor
7657b69387 Build and use our own etherpad image
We need to install the ep_headings plugin. We've got a hack in place
to do it in prod, but it seems like every time we npm install the
plugin in ansible, we need to restart the etherpad container
for $reasons.

It's cleaner to just build an image based on upstream with ep_headings
installed.

Change-Id: Id3b31bbabbbc5f6dcfe56486e48916384292f693
2020-04-13 08:07:09 -05:00
Monty Taylor
f04340afc6 Collect logs from manage-projects runs
We've looked at this one and it's ok.

Change-Id: Id83e36f235753e54361a429782356d628eec0d9c
2020-04-09 17:48:23 -05:00
Monty Taylor
8af7b47812 Get rid of all-clouds.yaml
We had the clouds split from back when we used the openstack
dynamic inventory plugin. We don't use that anymore, so we don't
need these to be split. Any other usage we have directly references
a cloud.

Change-Id: I5d95bf910fb8e2cbca64f92c6ad4acd3aaeed1a3
2020-04-09 16:44:20 -05:00
Clark Boylan
164b07688d Publish docs updates to docs.opendev.org
Continuing to transition things. One thing we need to do is publish our
opendev docs to docs.opendev.org.

Change-Id: I1f2e65af9910b5bb21e4fad6295ebdf2fd4bd341
2020-04-09 11:07:27 -07:00
Monty Taylor
579a0207b9 Trigger remote-puppet-else on hiera changes
These are important input data to puppet, so we should run the
job when they change.

Change-Id: I8398c8f4f6dc0ec1bb72ee040349bfc2232eed2e
2020-04-09 10:34:06 -05:00
Monty Taylor
a06a1329c5 Trigger everything on inventory changes
We can't really know what's changed in the inventory from a zuul
perspective, so it's safest to run everything if the inventory
changes. (This is especially important for being able to pulse
a backup setup, fwiw)

Change-Id: I13768ed2ee257c8175c378c58525a6c83654e83b
2020-04-08 10:46:55 -05:00
Monty Taylor
ee67e26b4a Remove ansible-cron role
Once the previous patch has landed and run on system-config
then we can remove the role altogether.

Change-Id: I14fef0e178544d71390a6601915c745647435ab9
2020-04-08 10:46:55 -05:00
Monty Taylor
c8aeaf8fd1 Run remote-puppet-else in zuul
Wow. It's the last one!

Change-Id: Ib064c8412a3df28790b9ef51d88c50d43de1842d
2020-04-08 10:46:52 -05:00
Monty Taylor
e7342c7e3e Run AFS in zuul
Change-Id: I5ea46cbf124006e38c045e2f93a19d33a9c3db91
2020-04-08 10:46:32 -05:00
Monty Taylor
4641a79aec Run gitea in zuul
Change-Id: Iaebcc1babcfc46c7765ab64f5e826f9e727a9298
2020-04-08 10:45:31 -05:00
Monty Taylor
6c2c31bcd5 Run review and review-dev in zuul
Change-Id: I6fa322110c729b43e0ca30f9e4fb38048246564c
2020-04-08 10:45:31 -05:00
Monty Taylor
1e79b848cd Rename ansible_forks to infra_prod_ansible_forks
ansible_forks is a magic var, so when we try to set it ansible
is overriding it.

Change-Id: I3ae45b28d545571f04b02927c3f6c97f145743bc
2020-04-08 10:45:29 -05:00
Monty Taylor
513da85123 Depend on infra-prod-update-system-config
It's not infra-prod-service-update-system-config.

Change-Id: Ic9a23a5954460e88d3d5d4ffaf0c9e3dafff2c66
2020-04-08 08:03:09 -05:00
Monty Taylor
38484fcc3b Add sytem-config-update and remove other deps from hourly
We need to make sure we're updating system-config before running
the playbooks, but we don't need to do the other things on the
hourly timer.

Change-Id: Ic2ddb8108626f629664709727b3f7eeaf48ddce8
2020-04-07 16:30:31 -05:00
Monty Taylor
9eaf6d1a8f Update typo on infra-prod-service-letsencrypt
We've got infra-prod-letsencrypt. That's not a thing.

Change-Id: I8eafdaace521e7aa3e7b2eb3969303c77b7b79c9
2020-04-07 13:45:31 -05:00
Monty Taylor
907bb41b31 Run zuul and nodepool related deploys hourly
We can't trigger these on zuul image publication yet, so to not
regress from the hourly run_all cron, run a few of the jobs
hourly.

Depends-On: https://review.opendev.org/717063
Change-Id: I9d502d368c33306ee32da572db1fe8ccd8b066b3
2020-04-07 13:41:27 -05:00
Zuul
39dbc4f329 Merge "Run zuul-preview in zuul" 2020-04-07 18:06:49 +00:00
Zuul
b3b8391f45 Merge "Make a new dockerized etherpad.opendev.org" 2020-04-07 18:06:48 +00:00
Monty Taylor
b23515c623 Make a new dockerized etherpad.opendev.org
Upstream likes building the settings file into the image, but that's
less exciting, let's bind-mount ours in.

Depends-On: https://review.opendev.org/717491/
Change-Id: Ia1894d884ef2a84e1282345b77fe07bf8898f367
2020-04-07 11:10:57 -05:00
Monty Taylor
3e175c40d0 Disable log collection pending further auditing
We need to default to false so that we can audit logs and then
enable them individually.

Change-Id: Iadb1f8f56ef43d1f35ca080318af22008525b643
2020-04-06 13:23:25 -05:00
Zuul
ddb9a92435 Merge "Run registry in zuul" 2020-04-03 22:08:38 +00:00
Zuul
cfc58619af Merge "Run backup in zuul" 2020-04-03 22:08:36 +00:00
Zuul
b203446174 Merge "Add file matchers for roles used via include_role" 2020-04-03 22:08:34 +00:00
Zuul
6b6104b8e8 Merge "Run static in zuul" 2020-04-03 22:08:32 +00:00
Zuul
f4fd151f27 Merge "Run mirror in zuul" 2020-04-03 22:08:30 +00:00
Zuul
272f320989 Merge "Run nameserver in zuul" 2020-04-03 21:55:17 +00:00
Zuul
2ea1ba1987 Merge "Run mirror-update in zuul" 2020-04-03 21:55:15 +00:00
Zuul
2a0274dff5 Merge "Run meetpad in zuul" 2020-04-03 21:51:44 +00:00