For reasons explained in [1] Debian's lsb_release.py on bullseye is
falling back to probing "apt-cache policy"
When (as currently), stretch is the testing release,
/etc/debian_version contains "stretch/sid", as shipped by
base-files. It is therefore impossible to rely on that file to
differentiate between a host running testing or unstable without
asking apt what is actually preferred when installing packages
(through parsing `apt-cache policy`). That's how `lsb-release --
codename` returns "sid" _xor_ "stretch".
The problem is, this parses the output of "apt-cache policy" which
fails for two reasons; firsly we have cleared out all the cache files,
so our hosts return anything until "apt-get update" is run, but
secondly because our mirrors do not have a "label" that matches in
this code at [2]
e.g. what we get out of "apt-cache policy" is
500 https://mirror.dfw.rax.opendev.org/debian bullseye/main amd64 Packages
release o=Debian,n=bullseye,c=main,b=amd64
origin mirror.dfw.rax.opendev.org
which is missing a "l=" field to make this parsing recognise it as a
valid source.
The label is set by reprepro [3]
Label
This optional field is simply copied into the Release files.
Add a label to make our mirrors look more like regular mirrors.
[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=845651
[2] https://sources.debian.org/src/lsb/11.1.0/lsb_release.py/#L191
[3] https://manpages.debian.org/stretch/reprepro/reprepro.1.en.html
Change-Id: Id705acbb3a01f43ae635a24fa3c24d0a05bdaa16
Starting in bullseye, Debian's security suite will add -security to
dist codenames, meaning we have stretch, buster, and
bullseye-security entries. Looks inconsistent, but is actually
correct.
Change-Id: I34806145f099868c2cdd95893b69cb1f4915f56f
Call `reprepro export` to always recreate indices, even for empty
dists. This is sort of a shotgun approach, local testing on the
server indicates it increases total time of a noop update by ~5.5
minutes for the "debian" repo, which is by far the worst case of
anything we mirror.
If this proves problematic, we can engineer a more targeted solution
to check for empty dists and only export those.
Change-Id: I7e39e427e1941f055fae0408e4c1f2a2f2b35547
This converts the reprepro mirror script to use the common functions
for timestamps and vos release. This function ssh's to the AFS server
and runs vos release directly there, avoiding many issues with
kerberos timeouts. This has been working successfully for the rsync
mirrors. This will also send stats back so we can keep an eye on the
timing.
Change-Id: I1be29f2d9ecaad03b22c87819e5ae8d16c4f177e
I missed these in the original port. For some reason we are
installing these directly from upstream keyservers in the puppet,
rather than from files like everything else.
Change-Id: Ie1fa956b96f3e6d091b3ffcaab5e0be370da8fc7
This converts the reprepro configuration from our existing puppet to
Ansible.
This takes a more direct approach; the templating done by the puppet
version started simple but over the years grew several different
options to handle various use-cases. This means you not only had to
understand the rather obscure reprepro configuration, but then *also*
figure out how to translate that from our puppet template layers.
Here the configuration files are kept directly (they were copied from
the existing mirror-update.openstack.org) and deployed with some light
wrapper tasks in reprepro/tasks/utils which avoids most duplication.
Note the initial cron jobs are left disabled so we can run some manual
testing before letting it go automatically.
Change-Id: I96a9ff1efbf51c4164621028b7a3a1e2e1077d5c
