When we added Apache as a filtering proxy on our Gitea backends in
order to more easily mitigate resource starvation, we did not set
any tuning to tell it when to recycle worker processes. As a result,
backends may continue serving requests with workers which pre-date
certificate rotation. This problem has also become more broadly
prevalent throughout our services with the introduction of Let's
Encrypt's 3-month certificate expirations as compared to our
previous 2-year certificates.
Add the same MaxConnectionsPerChild tuning to our Gitea backend
proxies as we use for our static sites and mirror servers.
Change-Id: I77d89385178a30f7dc5d04bedd0ab3772865c09f
The sync to our new ORD replica has completed and we're back to the
typical vos release cadence for this volume again.
This reverts commit 542c898021af20f4ad48fa04b78b65c8f6fff0b6.
Change-Id: I4bb2ddcc46c6c56c7124acc52dce6a60da1662b2
We're in progress replicating the AFS volume for tarballs to a
remote location for added redundancy, but this is blocking updates
of all the read-only replicas until it completes and we're unsure
how long that will take. In the meantime, serve content from the
writeable path instead of the read-only replica path so we're not
stuck with outdated content on the site.
Change-Id: I6e0333bdb9717a724fd29adffc3df6e6c5da1558
Starting in bullseye, Debian's security suite will add -security to
dist codenames, meaning we have stretch, buster, and
bullseye-security entries. Looks inconsistent, but is actually
correct.
Change-Id: I34806145f099868c2cdd95893b69cb1f4915f56f
Call `reprepro export` to always recreate indices, even for empty
dists. This is sort of a shotgun approach, local testing on the
server indicates it increases total time of a noop update by ~5.5
minutes for the "debian" repo, which is by far the worst case of
anything we mirror.
If this proves problematic, we can engineer a more targeted solution
to check for empty dists and only export those.
Change-Id: I7e39e427e1941f055fae0408e4c1f2a2f2b35547
The OpenEdge cloud has been offline for five months, initially
disabled in I4e46c782a63279d9c18ff4ba2944c15b3027114b, so go ahead
and clean up lingering references. If it is restored later, this can
be reverted fairly easily.
Depends-On: https://review.opendev.org/783989
Depends-On: https://review.opendev.org/783990
Change-Id: I544895003344bc8202363993b52f978e1c07d061
Adding this key allows us to safely rsync data in a R/O fashion from
the production server to the new server and will be useful as we
deploy review02.opendev.org. The key is hard-coded for one on the new
server.
Change-Id: I227876afafcb48715324ca35afdc0bff2492b29a
This doesn't install of Focal; moving forward we'll either use H2 or a
container database. Just remove this package for now.
Change-Id: I69cdcdddc1ba0e0cf4ef5f8ba705bcd3a2afa689
This was missed during recent updates; this UserList needs to be on
all servers to allow bos, vos and backup commands.
Update the documentation to reflect the centralised copy.
Change-Id: I8ada3d5035bb7ef77b19ce6aaffb48335974a124
This picks up a variety of bug fixes and ensures we're keeping up.
The diff of the template files we modify between gitea v1.13.1, v1.13.4,
and v1.13.6 is empty. The diff between our modifications at v1.13.4
looks about how I would expect (implying that v1.13.6 is also fine).
Reviews should double check though.
We also add in setup for the system-config repo on the test gitea as
this will give us something to look at for verification purposes.
Change-Id: Idb3568a9d287a2d46d568ab7d8d3a7108739d23e
review02.opendev.org is a much larger replacement server for review01
provided by Vexxhost. It is up and running, with gerrit2 volume
attached and DNS entries.
This adds it to the staging group with no replication and a local h2
database configured for initial bringup. There's quite a bit to
consider for full migration, but this will let us start experimenting.
Change-Id: I3638a5c0c7028dcc800ada42431b75395cff0c42
This moves the haproxy containers to syslog with tags that will get
each container's output filtered into /var/log/containers.
Change-Id: I65294baec3c092ede5ee97856d8d879174b0d8d4
The dstat-logger service puts a lot of info into the syslog/journal.
The --output command to write CSV files doesn't appear to suppress the
console output, and I can't see a flag to make it do that. So
redirect the stdout to /dev/null.
Change-Id: Ib99f8199ebc3c9d89c2b3aa92dff5ff298d03e45
Create a review-staging group so we can bring up a new server but
avoid running the project-management steps on it.
Change-Id: I93d2a36edcd58a48a36031f0692be3273a36f07c
With our increased ability to test in the gate, there's not much use
for review-dev any more. Remove references.
Change-Id: I97e9865e0b655cd157acf9ffa7d067b150e6fc72
Now that the update-bug script no longer tries to update bug
assignments, it's possible to run it on patchset-created events
again. Go back to doing that until someone has time to build a
suitable replacement for it.
This partially reverts commit
1ccf5e68e51815479381a941fd9cf4f469498c6d.
Change-Id: Idf589eb818d208d65d1f1430ddec962b015165c0
Depends-On: https://review.opendev.org/782538
In today's weird corner-case issue; when running under cron,
SHELL=/bin/sh ... which doesn't really matter (this script is run
under #!/bin/bash) *except* that "sudo -s" is obeying SHELL and
consequently the in-line script here fails under cron, but not when
run interactively. Just set SHELL=/bin/bash for consistency.
Change-Id: Ic8584b90fea8382f7a7d294b98a0a3689bfc981b
We found a bug in master which will prevent us from merging a fix;
downgrade the scheduler to 4.1.0 to get that in.
Change-Id: Ie9ad75177ab58b34e20cafab496ba7af6f082551
When we cleaned up the puppet in
I6b6dfd0f8ef89a5362f64cfbc8016ba5b1a346b3 we renamed the group
s/refstack-docker/refstack/ but didn't move the variables and some
other references too.
Change-Id: Ib07d1e9ede628c43b4d5d94b64ec35c101e11be8
This reverts commit f64b935778074191337672c739978975ee4eda68.
There must be more required to make this work.
Change-Id: I21e1d45fabca10ee93e7c87449fc15f3fd691b8b
This switches desktop clients to using xmpp over websockets instead of
BOSH. The mobile clients continue to use BOSH. Apparently this provides
better performance and is the default expectation of the upstream docker
images. We had disabled it prior to get back to a working state when we
weren't setting this variable at all.
After looking at configs on the docker images I expect that enabling
this explicitly will work (the problem before was we neither enabled nor
disabled it and the images can't handle that case). If that isn't the
case we can always revert.
Change-Id: I59c9fe75a0860782beb8864ff3bd9622b35381c1
This removes an unused letsencrypt dir bind mount for jitsi-meet web
that was causing confusion (we run letsencrypt out of band and put the
certs in the correct dir so we don't need this specific bind mount).
We also remove the now unused config.js config file from the role.
We stop managing the default nginx config and instead rely on the
container provided template. To properly configure http redirects we
set the ENABLE_HTTP_REDIRECT flag in the env var file.
Finally we update the README file with a bit more info on how this all
works.
Change-Id: Iecb68c9855b5627d25f8bb586b0e6f366f1c80ab
