Recently the hosted database that translate.openstack.org uses was
upgraded to MySQL 5.7. It turns out there was a breaking
security-related change to MySQL that means you require PROCESS
privileges to dump the DB. I guess the migration didn't do this, and
via the RAX hosting UI there's no easy way to add this (there's no
root user available by default; perhaps this is a corner case of the
upgrade process).
Rather than trying to figure out the backend, to work around this use
--no-tablespaces to the dump command. We just need a dump of the
zanata tables as backup, so this is fine.
Change-Id: Idaa35249fe15867103fa7b70dbdc7d135c5e1396
There are two issues in the zuul merger and executor shutdowns. The
first is that `docker-compose ps -q` will report exited containers
unlike `docker ps -q`. This means we may try to exec into a non running
container which is an error. Handle this by checking the error message
and proceeding if the 'is not running' string is present.
The second issue is a race between stopping a container and running an
exec in the container. If a container stops while an exec is running in
it that exec appears to be treated with some equivalent of kill -9. The
result is the exec returns 137. While theoretically possible for both
executor and merger graceful stop command we seem to only hit this with
the merger so we handle exit code 137 for the merger only. This way
we'll get info if the executors start running into this too.
Change-Id: Ia6dc2d7e397631d72968ffa89c4492b803c89c47
In the graceful shutdown for mergers and executors if we skip the docker
exec to stop the container we also need to skip the docker wait. The
reason for this is docker wait exits with an error code if not provided
with any arguments to wait for.
Change-Id: Id09666ee23e1a9599d477b63a89559e4ab1d21bf
Ansible syntax got me. When I updated the apt tasks to retry on apt/dpkg
locks I overindented the register, delay, retries, and until parameters.
These are to the task not the module.
Change-Id: I955d96b5467597503e0e5563e37ffa736ef2fcdc
This is the first minor Gitea update we've seen in a long time with
updated template files. I've updated the header.tmpl file to match
1.17.2's content.
Full changelog can be seen at:
https://github.com/go-gitea/gitea/blob/v1.17.2/CHANGELOG.md#1172---2022-09-06
Change-Id: I087f13beb64e2a3b607f1f60e2d3841567845543
The way the currently graceful stop tasks are written for zuul expects
there to always be a running zuul container to exec into. There are
situations where there may not be a running container in which case
there is nothing to stop. Avoid this being an error by checking if the
containers are running before execing into them. If no containers are
running then we'll noop the docker exec step allowing the rest of the
ansible tasks to continue.
Change-Id: I6c47147a589ae12cc33e37e40e49673396d120f7
The zuul_reboot playbook runs on each zuul server at what essentially
become random times based on how long the previous servers took to be
updated. We have seen this result in our apt tasks colliding with
unattended upgrades on the server.
Latest ansible would let us workaround this using the lock_timeout
parameter to the apt module, but the version we use on bridge does not
support this parameter. Instead we check the failure message for
'Failed to lock apt for exclusive operation' and if present we retry. We
wait 30 seconds between retries and will perform up to 40 attempts for a
total of 20 minutes of waiting. This method should also be forward
compatibile with new Ansible.
If the lock is held for longer than 20 minutes it likely implies
something has gone wrong and we will need to perform manual intervention
anyway.
Change-Id: I3171838a30e3ea496bb08f8b6ab1c95755b2ff3c
Normally this isn't an issue because we run logrotate more frequently
than our weekly cron to upgrade and reboot zuul. But if you need to
manually run the playbook and are referring to the crontab entry to
determine how to run the playbook then the resulting command could
truncate a recent run. Simply append to the file in all cases to avoid
this.
Change-Id: I393741317cccaf447912b1f1517e846c32ee7677
For some reason, the JVB servers now seem to tell clients to connect
to 8443/tcp on localhost rather than the actual server. It seems it
wants to build the URL based on the PUBLIC_URL envvar, but we
previously did not pass that through to the JVB containers. Add it
to their configuration so they'll have it available.
Change-Id: I10c761105490a72c4eb9ac0b08a304b7d5d1e18c
It appears upstream container init now copies this from defaults,
overwriting our modifications. Shadow the one in the container with
ours so it gets copied into the eventual destination.
Also switch back to the old muting variables we were using before,
since the new "with" bools seem not to work (still worth looking
into later).
Change-Id: I7e91e82e6f91b44c5c7eb1406ba0c64d30e6b8ff
The zuul pipeline reporter for merge-failure has been renamed to
merge-conflict. The old name has been depreacted and will be removed in
a future release. Update our examples to match Zuul's current
expectations.
Change-Id: I1f9effa311163d942171e35ba65fafa25245e9d2
Bring our 5 configs into line with current upstream versions
(jitsi-meet_7648 and stable-7648-4 tags from the jitsi-meet and
docker-jitsi-meet repositories respectively). Attempt to preserve
most of our earlier overrides:
* configure Etherpad integration
* disable background blurring
* disable watermarks
* open shared document on join
* start with audio and video muted
* redirect HTTP to HTTPS
* disable XMPP WebSockets
* disable P2P connections
* templated credentials
* templated unique JVB server identifiers
Drop any options we previously set which later became defaults (like
useRoomAsSharedDocumentName or UTC as the TZ). Identify the upstream
repo and tag on which each file is based. Stop claiming Firefox is
not recommended, now that the default configuration adds a pre-join
page which helps browsers realize they should not treat the audio
stream as unsolicited. Switch to newer vars for muting audio and
video as a boolean rather than at a participant threshold.
Update the docker-compose files to use the stable tag instead of
latest, since upstream seems to just stop refreshing the latest tag
far too often. Clean up extra envvars we were setting for JVB which
we didn't pass through to the containers.
Change-Id: I1e5a3836917f3d90ad7dd1c0771871740fda3cda
This reverts commit cc2dd16d3a7194a4185ad6e1da854cb4fde01b1c.
Reason for revert: rax mirrors not synched for 15 hours and causing
issues, facebook mirror is up to date so let's switch to it.
Change-Id: Iaf94540f22e2b49c74ab0704ac94fd1554ce5bbc
Related-Bug: #1988397
Refstack does not work with cryptography 37.0.0. In debugging how this
passes testing I noticed that Refstack's tox.ini uses openstack
constraints. Arguably it shouldn't do this as it isn't part of
coinstalled opensdtack services, but since this is how they test things
we should mimic it in our deployment.
Update the Dockerfile to copy in the upper-contraints file to the
/tmp/src dir which the assemble script should honor.
Change-Id: I4603702982e5c03b4a1d110363787b1f04887a29
We do this to prevent lookups to third parties for information in this
case avatar info. We should verify this doesn't break local avatar
storage usage which we manage directly.
Change-Id: I612bf1629bd211ed14203cc9e39f34ba0be041bf
Please carefully review the changelog:
https://github.com/go-gitea/gitea/blob/v1.17.1/CHANGELOG.md
and ensure that we've properly addressed the items listed there.
I have listed the breaking changes list here and any actions we've taken
or justification for why they don't affect us:
* Require go1.18 for Gitea 1.17 (#19918)
We were already using go 1.18.
* Make AppDataPath absolute against the AppWorkPath if it is not (#19815)
Path is already absolute:
playbooks/roles/gitea/templates/app.ini.j2:APP_DATA_PATH = /data/gitea
* Nuke the incorrect permission report on /api/v1/notifications (#19761)
This has to do with how that api endpoint returns permissions. We
don't use this anywhere as far as I can tell.
* Refactor git module, make Gitea use internal git config (#19732)
In the gitea container /data/git/.gitconfig is present but we don't
appear to manage this in system-config. I think that means this
change is a noop for us as gitea will move its managed .gitconfig
from /data/git/.gitconfig to /data/git/repositories/.gitconfig.
I expect the contents to be the same since gitea must be managing
the file old content today.
* Remove RequireHighlightJS field, update plantuml example. (#19615)
This was a flag that toggled syntax highlighting on and off as best
as I can tell. The default is to just have it turned on and we don't
check the flag in any of our templates.
* Increase minimal required git version to 2.0 (#19577)
Debian Bullseye ships with 2.30.2-1.
* Add a directory prefix gitea-src-VERSION to release-tar-file (#19396)
They were tarbombing people and their tarballs extracted into the
current dir. They now no longer do that. We build from git so this
doesn't affect us.
* Use "main" as default branch name (#19354)
We explicitly set the default branch name to master for both gitea and
gerrit. This should be a noop for us. Testing has been added to check
this.
https://opendev.org/opendev/system-config/src/branch/master/playbooks/roles/gitea-git-repos/library/gitea_create_repos.py#L129-L132https://opendev.org/opendev/jeepyb/src/branch/master/jeepyb/cmd/manage_projects.py#L488
* Make cron task no notice on success (#19221)
I'm not aware of us relying on any cron tasks or any cron task
notifications.
* Add pam account authorization check (#19040)
We don't integrate with pam so the change in behavior to check
authorization does not affect us.
* Show messages for users if the ROOT_URL is wrong, show JavaScript errors (#18971)
This message shows up in CI because ROOT_URL is https://opendev.org
but we access gitea in testing via localhost. I don't think this
is worth fixing. Its a good reminder that the instance is a test
instance.
* Refactor mirror code & fix StartToMirror (#18904)
We don't mirror repos with gitea. Should be a noop for us.
* Remove deprecated SSH ciphers from default (#18697)
hmac-sha1-96, diffie-hellman-group1-sha1, and arcfour{128,256} are
removed. The only ssh user is gerrit's replication. MINA should
be able to support more modern ciphers and be fine.
* Add the possibility to allow the user to have a favicon which differs from the main logo (#18542)
Previously, logo.svg was used as the favicon.svg and gitea only fell
back to favicon.png if the browser couldn't so the .svg. But now they
want to support users having different logo.svg and favicon.svg. This
necessitates explicitly adding a favicon.svg. Something we already do.
Details at https://github.com/go-gitea/gitea/pull/18542
* Update reserved usernames list (#18438)
This shouldn't be a problem for us as we don't have regular users and
gerrit is not a reserved name.
* Support custom ACME provider (#18340)
We run ACME with LE out of band. This doesn't affect us.
* Change initial TrustModel to committer (#18335)
This changes the signed commits trust model from collaborator
to committer. THis won't affect us as we aren't maintaining trusted
keys. But basically this now shows if the signed commit by the
committer matches the committer's key.
* Update HTTP status codes (#18063)
This changed redirect HTTP codes from 302 to 307. Shouldn't
affect us.
* Upgrade Alpine from 3.13 to 3.15 (#18050)
We build on Debian and not alpine. The alpine nodejs version did
change from 14 to 16 in this change and we've updated to match.
* Restrict email address validation (#17688)
If we had real users this may pose a problem as they are limiting
the set of emails gitea would accept to a smaller set than they
accepted before. Also fewer than actually allowed by email. But
we don't have real users so this should be fine.
* Refactor Router Logger (#17308)
This streamlines and improves the log format of some of the gitea
logs. We aren't automatically processing these logs today so this
shouldn't have a major impact on us.
Additionally this release adds a new git.HOME_PATH setting to set the
location for writing out git configs and potential gnupg configs. We
should be fine to let gitea write this content out to the default path,
but there is potential for this to impact our ssh daemon.
Changes made include:
* Minimal updates to web templates to match 1.17
* Updating nodejs to v16 as v14 failed to build gitea
* Disabling the new enabled by default "packages" feature
* New test to check repos have a master branch by default instead of
Gitea's new default of main.
Change-Id: I88105eccd118e3daca72f0b86a6b351c35e37413
We've seen CI systems consume all of our threads which causes the web UI
to become non responsive. To address this increase the number of httpd
threads from 100 to 150. Note that we do not modify sshd.threads beacuse
sshd.threads determines the max number of git requests across both ssh
and http.
In theory what this means is that httpd has an additional 50 threads to
process non git requests (for example web UI requests) which will
hopefully keep that responsive even if git requests are max'd out.
It is possible that we also need to increase the sshd.threads value to
handle those git requests, but we will start by modifying one config
value at a time. If we do bump sshd.threads we should increase
httpd.maxThreads to give it that additional headroom.
Finally, I believe this is likely to be safe as we doubled the size of
our Gerrit server when we moved it to vexxhost. The old server was
pretty well maxed out though so increase these values on the new server
slowly and monitor the results.
Details on the configuration can be found at:
https://gerrit-review.googlesource.com/Documentation/config-gerrit.html#httpd
Change-Id: I57a1e248c3c01597bb29c7afc304688e834a64cc
Include the latest refstack changes in the image.
The newest changes include a fix for py3.10 compatibility
and new 2 features - support for 2 new add-on programs.
Change-Id: I9bb0b51ac99af756744ec65cc4d35c2d9a2fb037
Requested by Alvaro Soto, one of the OID Mexico organizers.
Change-Id: I84c3502a0ce7258f9bea7da8ca6858d4cc4a487c
Depends-On: https://review.opendev.org/852922
We must have missed this, I noticed when it didn't run on the gate job
for I949c40e9046008d4f442b322a267ce0c967a99dc
Change-Id: I62c5c0f262d9bd53580367dc9f1ad00fe7b6f6f2
Facebook mirror is out of sync for some days, so i'm proposing to use
rackspace one. This reverts [1] as it seems it is accepting rsync
connections properly.
[1] https://review.opendev.org/c/opendev/system-config/+/824829
Change-Id: Ic0076191157be8947f62ce18d5dd37f1f0ac3337
We still have some Ubuntu Xenial servers, so cap the max usable pip
and setuptools versions in their venvs like we already do for
Bionic, in order to avoid broken installations. Switch our
conditionals from release name comparisons to version numbers in
order to more cleanly support ranges. Also make sure the borg run
test is triggered by changes to the create-venv role.
Change-Id: I5dd064c37786c47099bf2da66b907facb517c92a
This is the latest 1.1.18 release, and from the changelog there
doesn't seem to be anything important we need to take into account
from 1.1.14.
Just as a note the 1.2 series is released, but this requires much more
thought when updating.
Change-Id: I949c40e9046008d4f442b322a267ce0c967a99dc
Many of our tests are actually running with a timeout of 3600; I think
between a combination of bumping timeouts for failures and
copy-pasting jobs.
We are seeing frequent timeouts of other jobs without this,
particularly on OVH GRA1. Let's bump the base timeout to 3600 to
account for this. The only job that overrides this now is gitea,
which runs for 4800 due to it's long import process.
Change-Id: I762f0f7c7a53a456d9269530c9ae5a9c85903c9c
