16076 Commits

Author SHA1 Message Date
Zuul
2e2be9e687 Merge "Fix zuul.conf jinja2 template" 2020-04-25 15:28:37 +00:00
Zuul
ec84c8563e Merge "The package is libjemalloc1" 2020-04-25 15:28:34 +00:00
Zuul
c07d86d7ce Merge "Use pip3 to install zuul on executors" 2020-04-25 15:24:00 +00:00
Clark Boylan
18568882b8 Fix zuul.conf jinja2 template
This adds a necessary newline, removes port numbers, and sets the
executor ssh key to the correct path.

Change-Id: I6b4afa876b6cd7d8f87cc35bc51b4e9d6e31ee2b
2020-04-24 18:15:31 -07:00
Monty Taylor
b7c0c0f8ce The package is libjemalloc1
When we install packages on ubuntu, we should use their actual
package names rather than incorrect or otherwise fictional
package names.

Also, fix the hostname in the test job - because when we don't
do that, we don't run all of the roles, and thus we don't
catch these things.

Change-Id: I18e676ef0fe343513db4c8ad7e340ee45092c0a3
2020-04-24 18:15:31 -07:00
Clark Boylan
6570eda366 Use pip3 to install zuul on executors
The executors aren't using docker container images yet due to conflicts
with bubblewrap. This means we are still installing it directly on the
host using pip. Unfortunately we were using `pip` before which maybe
install to python2 and zuul doesn't run under python2. Address this by
explicitly using pip3.

Change-Id: I2ec551e8207e29ca420b09b8818154b9c32b47cf
2020-04-24 18:15:31 -07:00
Zuul
1c3a4520ef Merge "Remove an extra backslash" 2020-04-25 00:24:37 +00:00
Monty Taylor
77e848b237 Remove an extra backslash
Change-Id: Ia027a2d51ed51ece224de07ef0d45a7ee95dc485
2020-04-24 18:04:53 -05:00
Monty Taylor
05b0587871 Add nodepool node key
Change-Id: I28ccb83fc984190b1ce8e3e18c5945209fcb2387
2020-04-24 17:54:50 -05:00
Monty Taylor
c1714ef918 Use the sync-project-config role in service-zuul
We can sync the project-config repo instead of cloning it on the
hosts.

Change-Id: Ic238d743f4dda0a3c04a7e3618a9d86d3c3e5371
2020-04-24 14:43:35 -05:00
Zuul
1febd9a0f2 Merge "Add service-zookeeper to nightly jobs" 2020-04-24 17:58:21 +00:00
Zuul
8bd18e9089 Merge "Actually run service-zuul playbook in prod" 2020-04-24 17:34:26 +00:00
Zuul
d3360a7d60 Merge "Remove two unused ansible vars files" 2020-04-24 17:17:33 +00:00
Zuul
f6d60a638d Merge "Use python3 for ansible on etherpad" 2020-04-24 17:09:05 +00:00
Monty Taylor
d4922ce29d Add service-zookeeper to nightly jobs
We missed adding a periodic job for it when we added it.

Change-Id: I560bf11a56ffb1cf67f89f3bc2749758385dd37e
2020-04-24 16:59:39 +00:00
Monty Taylor
535e1602c0 Actually run service-zuul playbook in prod
We added the playbook but not so much a job to run it.

Change-Id: I385b57f6a4499ab0c6b6874eced7e9fd34e71bc7
2020-04-24 11:39:56 -05:00
Zuul
b21a8e58cf Merge "Run Zuul using Ansible and Containers" 2020-04-24 16:31:42 +00:00
Monty Taylor
275ccd9b80 Remove two unused ansible vars files
Change-Id: I415b709399df28d4da55b8fa0a204110729e70a8
2020-04-24 10:44:32 -05:00
Monty Taylor
47684ec0f4 Use python3 for ansible on etherpad
We need to override this because it's erroneously picking up
python2 which on etherpad does not have everything it needs.

Change-Id: Ic127f505740f9ac66abb8754356e04acde66dd85
2020-04-24 10:43:23 -05:00
Zuul
87756b9f3e Merge "Add new etherpad to cacti" 2020-04-24 15:02:38 +00:00
Zuul
3b48da5a74 Merge "Stop logging puppet to syslog" 2020-04-24 15:02:36 +00:00
Zuul
1b2d113c0f Merge "Split eavesdrop into its own playbook" 2020-04-24 15:02:34 +00:00
Monty Taylor
f0b77485ec Run Zuul using Ansible and Containers
Zuul is publishing lovely container images, so we should
go ahead and start using them.

We can't use containers for zuul-executor because of the
docker->bubblewrap->AFS issue, so install from pip there.

Don't start any of the containers by default, which should
let us safely roll this out and then do a rolling restart.
For things (like web or mergers) where it's safe to do so,
a followup change will swap the flag.

Change-Id: I37dcce3a67477ad3b2c36f2fd3657af18bc25c40
2020-04-24 09:18:44 -05:00
Zuul
e044023dc9 Merge "Split codesearch into its own playbook" 2020-04-24 13:57:03 +00:00
Monty Taylor
a4e7440069 Add new etherpad to cacti
Change-Id: I0e4e7415f2a237ee2ad425633a636e7c0c011b73
2020-04-23 17:18:36 -05:00
Zuul
81f9bfd996 Merge "Set AllowEncodedSlashes NoDecode on 8080 revproxy" 2020-04-23 20:12:15 +00:00
Monty Taylor
99aa528c83 Stop logging puppet to syslog
We run puppet with ansible now pretty much all the time. It's not
helpful for the puppet output to go to syslog on the remote host.
What's more helpful is for it to come back to the stdout in the
ansible playbook so that we can see it.

Also turn off ansi color from the output.

Depends-On: https://review.opendev.org/721732
Change-Id: I604081d5400bd53b8dda5a3a7685323c1443991b
2020-04-23 19:38:51 +00:00
Monty Taylor
9fd2135a46 Split eavesdrop into its own playbook
Extract eavedrop into its own service playbook and
puppet manifest. While doing that, stop using jenkinsuser
on eavesdrop in favor of zuul-user.

Add the ability to override the keys for the zuul user.

Remove openstack_project::server, it doesn't do anything.

Containerize and anisblize accessbot. The structure of
how we're doing it in puppet makes it hard to actually
run the puppet in the gate. Run the script in its own
playbook so that we can avoid running it in the gate.

Change-Id: I53cb63ffa4ae50575d4fa37b24323ad13ec1bac3
2020-04-23 14:34:28 -05:00
Zuul
2dcc8fc366 Merge "Add a uwsgi-base container image" 2020-04-23 16:14:20 +00:00
Zuul
b568391a4b Merge "Allow passing an arbitrary package list to assemble" 2020-04-23 16:07:41 +00:00
Zuul
e79de4587a Merge "Upgrade to gitea 1.11.4" 2020-04-23 15:37:44 +00:00
Zuul
e79be1e00a Merge "Use openstackclient from container" 2020-04-23 14:36:03 +00:00
Monty Taylor
08a807e188 Add a uwsgi-base container image
For things using the builder/base pattern providing uwsgi services.

Change-Id: Iaf6d31a3d119f6b7e87b54cda1969a9994110dad
2020-04-23 13:11:44 +00:00
Monty Taylor
967a055c5d Use openstackclient from container
We have multiple places, such as bridge, where we use openstackclient
as a utility. The current practice tends to be installing it into a
venv somewhere. On bridge there are 16 venvs in /root - and in some
of them but not all exist a working openstack client.

Since it's purely an API interaction tool, all it needs is the
clouds.yaml files bind-mounted in. The openstackclient project
now publishes a container image. Install a wrapper script for
calling it into /usr/local on bridge.

Change-Id: I4c64125d5dd3c89275e259a57b0efa0847ac6fc2
2020-04-23 07:46:28 -05:00
Monty Taylor
f3304c38d5 Allow passing an arbitrary package list to assemble
This lets you build an image to install a package or set of packages
without having to build a fake pbr install.

Change-Id: I6c171881283b89a14a532f31aedb33e223a0ea1f
2020-04-22 19:15:51 -05:00
Clark Boylan
2e2ee170f8 Fix rooted path to docker-compose
In places like crontab entries we use full paths to executables because
PATH is different under cron. Unfortunately, this meant we broke
docker-compose commands using /usr/bin/docker-compose when we installed
it under /usr/local/bin/docker-compose. In particular this impacted
database backups on gitea nodes and etherpad.

Update these paths so that everything is happy again.

Change-Id: Ib001baab419325ef1a43ac8e3364e755a6655617
2020-04-22 14:09:46 -07:00
Zuul
91a2024d6d Merge "Allow requesting a list of extras to install" 2020-04-22 20:30:43 +00:00
Monty Taylor
ba3a0eb775 Allow requesting a list of extras to install
We produce lists of extras requirements to install, but installing
them takes an extra step. Allow it to be done all at once by accepting
names of extras on the install-from-bindep command line.

Change-Id: Ie75b9a668569c759f78b70617e311ed6f025039a
2020-04-22 14:39:55 -05:00
Zuul
b4318aea19 Merge "Start mirroring focal, stop mirroring trusty" 2020-04-21 20:19:41 +00:00
Zuul
f2152478e2 Merge "Remove some extra bits from site.pp" 2020-04-21 20:18:57 +00:00
Zuul
0b46f403ec Merge "Rearrange set-hostnames and cloud-init removal" 2020-04-21 20:18:55 +00:00
Monty Taylor
d5c68c5131 Split codesearch into its own playbook
Make a service playbook, manifest and jobs for codesearch.

Remove openstack_project::server - it doesn't do anything.

Change-Id: I44c140de4ae0b283940f8e23e8c47af983934471
2020-04-21 13:18:28 -05:00
Monty Taylor
2f6483a236 Remove some extra bits from site.pp
We don't default to running anything in puppet anymore. Let's
get rid of these.

Change-Id: I4de06c4c29ac99ab23f24d69b546fefc0e2c2336
2020-04-21 13:18:28 -05:00
Monty Taylor
68b50ca05b Rearrange set-hostnames and cloud-init removal
In launch-node, we run two playbooks that aren't part of base.
One sets the system's hostname and removes cloud-init, the other
runs unattended update.

We need to run the hostname setting in our functional tests so
that the hosts behave as expected, but running the cloud-init
removal is a little weird, since our test nodes already don't
have it.

Make it so that set-hostname actually just sets the hostname,
and then run it in run-base. For running puppet, we need the
host to have the correct hostname.

Move cloud-init removal to the base-server role. Also move
the autoremove into base-server, since it's probably a nice
way to get rid of excess things.

Change-Id: I53cb8c515444a7d73b839e799c5794b067429daa
2020-04-21 13:18:24 -05:00
Zuul
fe7d38af63 Merge "Remove system-config-puppet-beaker-rspec-puppet-4-centos-7-infra" 2020-04-21 18:09:45 +00:00
Zuul
8128922ca3 Merge "Remove global variables from manifest/site.pp" 2020-04-21 17:05:55 +00:00
Zuul
5bb5cd9c30 Merge "Remove unused gerrit puppet things" 2020-04-21 17:05:53 +00:00
Zuul
f1732987f0 Merge "Remove old etherpad.openstack.org" 2020-04-21 17:05:50 +00:00
Zuul
fa67bbd003 Merge "Cleanup unneeded things post docker-compose upgrade" 2020-04-20 22:26:47 +00:00
Zuul
fbcc45401b Merge "Use real passwords for meetpad" 2020-04-20 18:29:43 +00:00