The config should use the full path to the config directory, append
/etc/reprepro to the job.
Currently all the reprepro jobs hash to the same start time because it
uses the hostname as a seed. Use the unique string name as the seed
so each job starts at a unique time.
Change-Id: If2745d0cd274f390dbff6337b7a44093b5919908
Logs show that the nameservers are being notified via ipv6 and
rejecting the request:
nsd[18851]: notify for acme.opendev.org. \
from 2001:4800:7819:104:be76:4eff:fe04:43d0 refused, no acl matches.
Modify the nsd ACL to allow the ipv6 of the master to trigger updates.
This is important for the letsencrypt process, where we need the
acme.opendev.org domain updated in a timely fashion so that TXT
authentication works.
Change-Id: I785f9636dd05e15b8ffd211845f439be7e8344a3
These volumes were removed with
I050f737521fa6837f3b6b52b8028a839a29f7bd2 but I forgot to remove them
from this list.
Change-Id: I6b7f4a3aef55627d523eca2183379dff15554046
Enable the Ansible based cron jobs, and disable the puppet host
versions to cut over the mirroring to the new server.
Change-Id: I0ffb1c484e64e67f5a5017dc3c3c8ebcdc3845c8
I missed these in the original port. For some reason we are
installing these directly from upstream keyservers in the puppet,
rather than from files like everything else.
Change-Id: Ie1fa956b96f3e6d091b3ffcaab5e0be370da8fc7
In converting this to ansible I forgot to install the reprepro keytab.
The encoded secret has been added for production.
Change-Id: I39d586e375ad96136cc151a7aed6f4cd5365f3c7
This writes out the ssh config so the backup process uses the right
key/user. Since we have a transition period where we have bup and
borg backups we need to make the borg config have unique markers, or
the two fight over the configuration block.
Change-Id: I5455da3f2829e2aa8e0c531193adbbeff4b4776d
We're using logrotate to keep a small number of db backups locally. We
write these backups to disk compressed. We don't want logrotate to
recompress them. This is unnecessary extra work.
Change-Id: Iafe1628ff421f47cf3e5cbee14998eeceb60be4c
This converts the reprepro configuration from our existing puppet to
Ansible.
This takes a more direct approach; the templating done by the puppet
version started simple but over the years grew several different
options to handle various use-cases. This means you not only had to
understand the rather obscure reprepro configuration, but then *also*
figure out how to translate that from our puppet template layers.
Here the configuration files are kept directly (they were copied from
the existing mirror-update.openstack.org) and deployed with some light
wrapper tasks in reprepro/tasks/utils which avoids most duplication.
Note the initial cron jobs are left disabled so we can run some manual
testing before letting it go automatically.
Change-Id: I96a9ff1efbf51c4164621028b7a3a1e2e1077d5c
We don't need to keep using the old 2.2 Order, Allow, Deny, Satisfy acl
primitives because we are now running Apache 2.4 everywhere. Stick to
these as they simplify understanding of acls by being consistent.
Change-Id: I9ed4edf15f206006fd79bdef298f8ed101a7a381
Docker has long planned to turn this off and it appears that they have
done so. Planning details can be found at:
https://www.docker.com/blog/registry-v1-api-deprecation/
Removing this simplifies our configs as well as testing. Do this as part
of good hygiene.
Change-Id: I11281167a87ba30b4ebaa88792032aec1af046c1
The user agent filter has been turned into a reusable Ansible role
containing a macro definition. Add that role and replace the
hard-coded copy of the user agent filter here with that
UserAgentFilter macro.
Change-Id: Ic24a38c93f0f68fab9ef1168de91ffad477fe13c
A copy of the filter used for our Gitea farm, this same activity has
been showing up on our tarballs.opendev.org site as well which is
consuming available connection slots for all vhosts on the static
server.
This is implemented as a macro so that it can be included into
additional vhosts, and put into a separate role so that it can be
added to all playbooks which need it. A subsequent change will add
it to the Gitea servers, eliminating the redundant copy there.
Change-Id: Ic2020b753076209f7708f76744fdf746bf933bd9
We don't need to keep using the old Apache 2.2 Satisfy ACL primitive
because we are now running Apache 2.4 everywhere. Stick to Require
as it simplifies understanding of ACLs by being consistent.
Change-Id: Ib2f7ea1909b9798279efc77a42b632e7129bd1d0
We found a couple of projects that were initially moved under "x/" but
then moved back under "openstack/" later. The original scripts didn't
take this into account (I5bf2ddf09b3df71a3428a8a0c535b131ecbc0bca has
been updated to note this).
The affected projects have been moved back manually on AFS, and this
corrects the website redirects.
Change-Id: I59ba05923ec5aa1ca8fed337b6384064b3038836
The existing filtered UAs seem to catch the bulk of the traffic but
there are a few common ones that are still sneaking through. Add four
new rules for these cases.
All three are MSIE variants from version 6 to 9. All old enough that we
should be able to do this safely without impacting real users.
Change-Id: I8ae59f38de8b30bd06e1643ddbccf81ea32858aa
When moving unofficial repositories out of the openstack git
namespace, compute-hyperv was included because it was not under the
governance of any official team or SIG. Later the OpenStack
Winstackers team adopted it and we moved it back into the openstack
git namespace.
More recently, when moving and redirecting tarballs site
content/URLs based on the original git namespace moves, we failed to
take into account that this project had moved back into the
openstack git namespace. Undo the redirect, and then the old
tarballs will be moved manually to match.
Change-Id: I208d3196ac38ccfbad6269a75848339c95e08c2b
Since we haven't used this anywhere yet, let's start with the latest
version.
Fix role matching for job too.
Change-Id: I22620fc7ade8fbdb664100ef6b6ab98c93d6104f
The title.svg logo for opendev and two jquery js files are no longer
managed by ansible nor do they appear to be in our docker image. They
appear to have been lost when we converted from puppet to ansible +
docker. Add them back in. We are also missing the icla html content (but
not the other clas) add this one back in.
We vendor the js contents even though in the past we copied them from a
git repo clone and a distro package installation. This way we don't have
unexpected surprises, record that the files are used, and can always
update them later.
Change-Id: I981b4b0f233ece45d03a80dc1724a4e496f66eb8
This matches the file, which got lost in my original script because I
didn't quote a $. Also add some quotes for better grouping.
Change-Id: I335e89616f093bdd2f0599b1ea1125ec642515ba
There appears to be a gitea bug that causes PATCH updates to projects to
fail when the cache is in a bad state for that project. We use PATCH
updates to a project to set the project descriptions. Since project
descriptions are not critical to gitea functionality (we weren't
updating them until last week) we can treat this as best effort and
ignore these failures.
We'll log these cases to aid in further debugging but continue on. The
next pass can retry.
Change-Id: I625bdc0856caaccb6b55931b0cdc6cf11a0bf3e1
Gitea has added a STACKTRACE_LEVEL config option to set which log level
will also generate stack traces when logging. We want them for at least
Error and above so set this to Error for now. In particular there seems
to be a commit cache issue which results in errors that having stack
traces for would be helpful to debug.
Change-Id: I0491373ef143dfa753c011d02e3c670c699d2a52
When we moved projects out of openstack/ into opendev/ we didn't also
move their tarballs.
This redirects affected old directories to their new per-tenant home.
See I5bf2ddf09b3df71a3428a8a0c535b131ecbc0bca for info on how this
list was generated.
Change-Id: Ib545a772ecfce475c1007f04c5b5145d375dae23
The upstream mirror may have private contents in dirs like .~tmp~/ or
snapshot/. We exclude these to avoid syncing problems when we don't have
permissions to read them.
Change-Id: I8d366f0e95667bfbe65f259877b13bd0d93cd877
Add the cron job that existed in puppet-graphite to cleanup old,
un-updated stats and directories.
Change-Id: Iac4676ae0ea1d5f1b96b6214ab6ab193c71a2d20
This is the storage-schemas configuration file currently deployed by
puppet-graphite. Apply it to the container so we maintain the same
retention, etc.
Change-Id: Ia733bf4a958a559ce3921094bb3f0875365157ce
