As of pip 22.0, its HTML parser no longer accepts any page which
doesn't start with the string "<!DOCTYPE html>" and, unfortunately,
Apache's mod_autoindex declares a very specific HTML 3.2 doctype
instead, causing pip to break any time our wheel cache is added to
its indices. The main index we generate has been updated with
https://review.opendev.org/826969 but we need this change to address
Apache's dynamically generated file lists for that site.
Configure Apache to supply a custom header file for file indices
within the /wheel/ subtree of our mirror vhosts, and alias it from
outside the docroot in order to reduce clutter of the top-level
directory index. Also instruct mod_autoindex to omit its own
document preamble which would otherwise include the original doctype
declaration. Note that this omits the header title and H1 level
headings from the resulting pages, but as these are only meant for
machine parsing anyway and not humans, it's a compromise to keep the
solution as simple and straightforward as possible.
Change-Id: Id71174954b13b80483256d37f773b781f4956c21
We've noticed that our mirrors will semi regularly have problems due to
old stale works. For example using old ssl certs or having connection
problems to round robin backend services. In all cases restarting the
service (killing old workers) seems to fix things. Try to force this to
automatically happen by setting a reasonable connection limit per worker
before we recycle them.
Change-Id: Ic377f48d1a5a3eecbcb183327c9255134c4364ab
This impelements mirrors to live in the opendev.org namespace. The
implementation is Ansible native for deployment on a Bionic node.
The hostname prefix remains the same (mirrorXX.region.provider.) but
the groups.yaml splits the opendev.org mirrors into a separate group.
The matches in the puppet group are also updated so to not run puppet
on the hosts.
The kerberos and openafs client parts do not need any updating and
works on the Bionic host.
The hosts are setup to provision certificates for themselves from
letsencrypt. Note we've added a new handler for mirror nodes to use
that restarts apache on certificate issue/renewal.
The new "mirror" role is a port of the existing puppet mirror.pp. It
installs apache, sets up some modules, makes some symlinks, sets up a
cleanup cron job and installs the apache vhost configuration.
The vhost configuration is also ported from the extant puppet. It is
simplified somewhat; but the biggest change is that we have extracted
the main port 80 configuration into a macro which is applied to both
port 80 and 443; i.e. the host will have SSL support. The other ports
are left alone for now, but can be updated in due course.
Thus we should be able to CNAME the existing mirrors to new nodes, and
any existing http access can continue. We can update our mirror setup
scripts to point to https resources as appropriate.
Change-Id: Iec576d631dd5b02f6b9fb445ee600be060f9cf1e
