1714 Commits

Author SHA1 Message Date
Zuul
9331c59029 Merge "dstat-logger: redirect stdout to /dev/null" 2021-03-30 23:57:29 +00:00
Ian Wienand
75ecf2cfbf gerrit: remove mysql-client-core-5.7 package
This doesn't install of Focal; moving forward we'll either use H2 or a
container database.  Just remove this package for now.

Change-Id: I69cdcdddc1ba0e0cf4ef5f8ba705bcd3a2afa689
2021-03-30 13:09:39 +11:00
Zuul
27d58d3b57 Merge "Add review02.opendev.org" 2021-03-30 00:48:54 +00:00
Zuul
39d8010055 Merge "openafs-server-config: install UserList" 2021-03-29 23:43:07 +00:00
Zuul
0800f4040e Merge "Upgrade gitea to 1.13.6" 2021-03-29 23:42:59 +00:00
Ian Wienand
ce7ef6536a openafs-server-config: install UserList
This was missed during recent updates; this UserList needs to be on
all servers to allow bos, vos and backup commands.

Update the documentation to reflect the centralised copy.

Change-Id: I8ada3d5035bb7ef77b19ce6aaffb48335974a124
2021-03-30 09:49:53 +11:00
Zuul
3180086559 Merge "Rename refstack group variables" 2021-03-29 21:33:02 +00:00
Clark Boylan
a08f65632f Upgrade gitea to 1.13.6
This picks up a variety of bug fixes and ensures we're keeping up.
The diff of the template files we modify between gitea v1.13.1, v1.13.4,
and v1.13.6 is empty. The diff between our modifications at v1.13.4
looks about how I would expect (implying that v1.13.6 is also fine).
Reviews should double check though.

We also add in setup for the system-config repo on the test gitea as
this will give us something to look at for verification purposes.

Change-Id: Idb3568a9d287a2d46d568ab7d8d3a7108739d23e
2021-03-29 11:03:59 -07:00
Zuul
d9723fda95 Merge "reprepo debian : fix line-ending" 2021-03-29 11:01:05 +00:00
Zuul
62ea8d1289 Merge "reprepro: add dist for Ubuntu UCA Wallaby" 2021-03-29 05:44:07 +00:00
Ian Wienand
9457b010f8 reprepo debian : fix line-ending
reprepro warns about lack of a trailing newline

Change-Id: I01c0cc2104cb1b3891ea55b6e4a3eab63885331a
2021-03-29 16:41:26 +11:00
Zuul
fff346ea32 Merge "Add Debian Bullseye to the reprepro config" 2021-03-29 05:30:21 +00:00
Ian Wienand
8541605595 Stop mirroring Fedora 31
Nodes were stopped with I0dde34ab005f48ac521d91e407ac437d3cec965f

Change-Id: Ib9cbc3f5388e668923ff7eadb3e81703875f64c0
2021-03-29 11:55:14 +11:00
Ian Wienand
525d5d1c19 Add review02.opendev.org
review02.opendev.org is a much larger replacement server for review01
provided by Vexxhost.  It is up and running, with gerrit2 volume
attached and DNS entries.

This adds it to the staging group with no replication and a local h2
database configured for initial bringup.  There's quite a bit to
consider for full migration, but this will let us start experimenting.

Change-Id: I3638a5c0c7028dcc800ada42431b75395cff0c42
2021-03-26 14:53:31 +11:00
Ian Wienand
d0f2e12f85 dstat-logger: redirect stdout to /dev/null
The dstat-logger service puts a lot of info into the syslog/journal.
The --output command to write CSV files doesn't appear to suppress the
console output, and I can't see a flag to make it do that.  So
redirect the stdout to /dev/null.

Change-Id: Ib99f8199ebc3c9d89c2b3aa92dff5ff298d03e45
2021-03-24 22:23:13 +00:00
Ian Wienand
163d5b6133 Create review-staging group
Create a review-staging group so we can bring up a new server but
avoid running the project-management steps on it.

Change-Id: I93d2a36edcd58a48a36031f0692be3273a36f07c
2021-03-24 11:40:33 +11:00
Ian Wienand
9f11fc5c75 Remove references to review-dev
With our increased ability to test in the gate, there's not much use
for review-dev any more.  Remove references.

Change-Id: I97e9865e0b655cd157acf9ffa7d067b150e6fc72
2021-03-24 11:40:31 +11:00
Zuul
c164b6f192 Merge "borg-backup-server: set SHELL for verification script" 2021-03-23 22:53:07 +00:00
James E. Blair
e2a141a9fa Revert "Downgrade scheduler to Zuul 4.1.0"
This reverts commit 3e3f86458d50e6c25dff42c8083a5b62e64a6e43.

Bugs should be fixed now.

Change-Id: If4279746074abfa42b8240d4e98dec59ab15d332
2021-03-23 12:12:18 -07:00
Mark Goddard
0ab854d833 reprepro: add dist for Ubuntu UCA Wallaby
Change-Id: I68357fb8353022c1c808f3a83a3e14da872be6c4
2021-03-23 10:09:36 +00:00
Ian Wienand
86ed1d74dd borg-backup-server: set SHELL for verification script
In today's weird corner-case issue; when running under cron,
SHELL=/bin/sh ... which doesn't really matter (this script is run
under #!/bin/bash) *except* that "sudo -s" is obeying SHELL and
consequently the in-line script here fails under cron, but not when
run interactively.  Just set SHELL=/bin/bash for consistency.

Change-Id: Ic8584b90fea8382f7a7d294b98a0a3689bfc981b
2021-03-23 14:53:56 +11:00
Zuul
aa4f1b4256 Merge "kerberos-kdc: quote some integers to avoid string/int confusion" 2021-03-22 22:56:26 +00:00
James E. Blair
3e3f86458d Downgrade scheduler to Zuul 4.1.0
We found a bug in master which will prevent us from merging a fix;
downgrade the scheduler to 4.1.0 to get that in.

Change-Id: Ie9ad75177ab58b34e20cafab496ba7af6f082551
2021-03-22 13:43:52 -07:00
Ian Wienand
aa94f2d831 Rename refstack group variables
When we cleaned up the puppet in
I6b6dfd0f8ef89a5362f64cfbc8016ba5b1a346b3 we renamed the group
s/refstack-docker/refstack/ but didn't move the variables and some
other references too.

Change-Id: Ib07d1e9ede628c43b4d5d94b64ec35c101e11be8
2021-03-19 16:01:46 +11:00
Zuul
b8874e4f51 Merge "kerberos-kdc: add database backups" 2021-03-19 00:06:59 +00:00
Zuul
eb78355fce Merge "borg-backup-server: fix verification run" 2021-03-19 00:05:49 +00:00
Ian Wienand
9ac0748802 kerberos-kdc: quote some integers to avoid string/int confusion
Change-Id: Ia7ed570c1a10feafcb4824dca10282b8866f5265
2021-03-19 10:37:49 +11:00
Clark Boylan
d0bc265c2a Revert "Enable jitsi-meet xmpp websockets"
This reverts commit f64b935778074191337672c739978975ee4eda68.

There must be more required to make this work.

Change-Id: I21e1d45fabca10ee93e7c87449fc15f3fd691b8b
2021-03-18 15:33:20 -07:00
Zuul
bf8e8916aa Merge "Enable jitsi-meet xmpp websockets" 2021-03-18 22:09:12 +00:00
Clark Boylan
0aa838ce16 Fix jitsi config.js
There were : used when we should have used =. Fix this.

Change-Id: Icb1e04d6e6e27726a12a1e49d53d9eb7c88d1a01
2021-03-18 13:43:39 -07:00
Clark Boylan
f64b935778 Enable jitsi-meet xmpp websockets
This switches desktop clients to using xmpp over websockets instead of
BOSH. The mobile clients continue to use BOSH. Apparently this provides
better performance and is the default expectation of the upstream docker
images. We had disabled it prior to get back to a working state when we
weren't setting this variable at all.

After looking at configs on the docker images I expect that enabling
this explicitly will work (the problem before was we neither enabled nor
disabled it and the images can't handle that case). If that isn't the
case we can always revert.

Change-Id: I59c9fe75a0860782beb8864ff3bd9622b35381c1
2021-03-18 11:59:14 -07:00
Clark Boylan
55f38141c5 More jitsi meet config cleanups
This removes an unused letsencrypt dir bind mount for jitsi-meet web
that was causing confusion (we run letsencrypt out of band and put the
certs in the correct dir so we don't need this specific bind mount).

We also remove the now unused config.js config file from the role.

We stop managing the default nginx config and instead rely on the
container provided template. To properly configure http redirects we
set the ENABLE_HTTP_REDIRECT flag in the env var file.

Finally we update the README file with a bit more info on how this all
works.

Change-Id: Iecb68c9855b5627d25f8bb586b0e6f366f1c80ab
2021-03-18 11:55:02 -07:00
Zuul
bb1b98623d Merge "Restore meetpad etherpad settings." 2021-03-18 18:04:35 +00:00
Zuul
c600c4a2cc Merge "Restore some meetpad settings we had previously set" 2021-03-18 16:59:09 +00:00
Zuul
4302bf3585 Merge "Manage jitsi-meet meet.conf as a template input for the container" 2021-03-18 15:54:14 +00:00
Zuul
99a05bdf75 Merge "Add kerberos-client group" 2021-03-18 02:43:59 +00:00
Ian Wienand
dc827de23d Add kerberos-client group
We duplicate the KDC settings over all our kerberos clients.  Add
clients to a "kerberos-client" group and set the variables in a group
file.

Change-Id: I25ed5f8c68065060205dfbb634c6558488003a38
2021-03-18 11:59:30 +11:00
Zuul
47fa6e0382 Merge "Add zookeeper-statsd" 2021-03-18 00:08:12 +00:00
Clark Boylan
c1bb5b52cf Restore meetpad etherpad settings.
This restores useRoomAsSharedDocumentName and openSharedDocumentOnJoin
config settings in our jitsi meet config.js. We had lost these settings
in the recent jitsi meet web container update. To restore them we
provide an alternative settings-config.js template to the container so
that when it generates its configs we get these vars included.

We stop managing the config.js file in /var/jitsi-meet/web to avoid
confusion with ansible replacing configs that may be used then.

Change-Id: I4d2bd77e03812695792cda2abb7f401288186f2c
2021-03-17 15:04:16 -07:00
James E. Blair
96bac7b486 Add zookeeper-statsd
This adds a program, zookeeper-statsd, which monitors zookeeper
metrics and reports them to statsd.  It also adds a container to
run that program.  And it runs the container on each of the
ZooKeeper quorum members.  And it updates the graphite host to
allow statsd traffic from quorum members.  And it updates the
4-letter-word whitelist to allow the mntr command (which is used
to gather metrics) to be issued.

Change-Id: I298f0b13a05cc615d8496edd4622438507fc5423
2021-03-17 14:52:31 -07:00
Clark Boylan
d410b43b59 Restore some meetpad settings we had previously set
This starts conferences with participant video muted and it disabled p2p
connections for calls between two participants. We had these settings
before but the recent container image update undid them.

Change-Id: I4152ace083f79116758020fbbcbbb96e39eef9ed
2021-03-17 13:24:38 -07:00
Clark Boylan
2ac695f628 Manage jitsi-meet meet.conf as a template input for the container
The jitsi meet containers want to generate configuration from the
templates found in /defaults on the container to config files in the
bind mounted /config (/var/jitsi-meet/ on the host side). This means
that the configs ansible is writing to /var/jitsi-meet are complete
ignored and overwritten by the container using its templating system and
env vars.

This is causing us problems because we would like to use a different
etherpad prxoy config in nginx to ensure the Host header is set
properly. To make this happen we bind mount in our own template file so
that the container can template what we want rather than what is found
in the image.

Change-Id: Ifdde66a01bb7e632fc19ca0a512216584f1ea9f0
2021-03-17 13:03:09 -07:00
Clark Boylan
75a64427a1 Improve meetpad env options for templating
The PUBLIC_URL is quoted which results in quotes ending up in our config
breaking etherpad base url setting in config.js. We remove the quotes as
they are not necessary.

We also remove the /p/ suffix from ETHERPAD_URL_BASE as this causes the
proxying to send extra /p/s to etherpad which results in problems.

Note these fixes appear to be necessary but are not sufficient to have
working meetpad proxying of etherpad. We also need to fix the nginx
meet.conf proxy settings to send valid Host heads. A followup change
will attempt to address that.

Change-Id: I0f59339a33267468ad5481858507a43cefa0021d
2021-03-17 12:47:43 -07:00
Clark Boylan
7b87c7c305 Disable xmpp websocket in jitsi meet config
We unforked our jitsi web container and discovered that etherpad doc
embedding was broken. In the process of debugging this the jitsi meet
services on meetpad were restart which pulled in newer configs which
expect ENABLE_XMPP_WEBSOCKET to be enabled by default. Unfortunately
this wasn't quite working for us. Explicitly disabling this seems to
make audio and video calling work again. But doc sharing isn't even
attempted now.

Let's get this fix in as audio and video are important then we'll keep
debugging the etherpad doc sharing problem.

https://github.com/jitsi/docker-jitsi-meet/issues/902 has details from
others that hit this problem.

Note that part of the issue here seems to be that nginx is using the
default configs in the container found at /default and not the configs
we bind mount at /config. This at least seems to be why the proxying for
etherpad documents is broken.

Change-Id: I03fa9d331e6825b3b953a3573c0dd43c7be478a4
2021-03-17 11:38:56 -07:00
Ian Wienand
e5a2354451 borg-backup-server: fix verification run
&>> is a bashism and not supported by sh, which cron runs the jobs
under.  Use >> instead.

Change-Id: I8e67f466887070fb1dedc403c53227c3ce1b2f1d
2021-03-17 15:09:57 +11:00
Zuul
77b1c14a9a Merge "Use upstream jitsi-meet web image" 2021-03-17 00:22:50 +00:00
Zuul
4524a92caf Merge "kerberos-kdc: role to manage Kerberos KDC servers" 2021-03-16 22:28:46 +00:00
Zuul
b133afedfd Merge "refstack: cleanup old puppet" 2021-03-16 22:21:03 +00:00
Ian Wienand
3052ff4935 kerberos-kdc: add database backups
Add a script to save a db dump to borg backups.  Add the primary KDC
to our backup list.

Change-Id: I32f4ebc1bb4c1952034aba43c75e4d2f85a1b6d3
2021-03-17 08:31:52 +11:00
Ian Wienand
c1aff2ed38 kerberos-kdc: role to manage Kerberos KDC servers
This adds a role and related testing to manage our Kerberos KDC
servers, intended to replace the puppet modules currently performing
this task.

This role automates realm creation, initial setup, key material
distribution and replica host configuration.  None of this is intended
to run on the production servers which are already setup with an
active database, and the role should be effectively idempotent in
production.

Note that this does not yet switch the production servers into the new
groups; this can be done in a separate step under controlled
conditions and with related upgrades of the host OS to Focal.

Change-Id: I60b40897486b29beafc76025790c501b5055313d
2021-03-17 08:30:52 +11:00