Ansible Galaxy appears to be served behind cloudflare and is currently
throwing 504 errors waiting for the backend to respond on /api/
requests. Since we know this is already not working and this is
preventing other changes from landing lets just go ahead and completely
disable the testing of this proxy.
We can always add the test back if and when effort is made to proxy the
new version of galaxy's api.
Change-Id: Iad35fc19ce4f8bb8ec3c57eb995b44fc5e61a06c
At some stage in the last 24 hours it looks like galaxy.ansible.com
changed and the current canary we look for, "Ansible NG", is no longer
present in the returned HTML:
$ curl -s https://galaxy.ansible.com/ ; echo
<!doctype html><html lang="en-US"><head><meta charset="UTF-8"><title>Ansible Galaxy</title><link rel="icon" href="/favicon.ico"><script defer="defer" src="/js/App.9bfa0e2736606eaddfe9.js"></script><link href="/css/App.4aff5598f0220c63b019.css" rel="stylesheet"></head><body><div id="root"></div></body></html>
$ curl -s https://mirror01.dfw.rax.opendev.org:4448/ ; echo
<!doctype html><html lang="en-US"><head><meta charset="UTF-8"><title>Ansible Galaxy</title><link rel="icon" href="/favicon.ico"><script defer="defer" src="/js/App.9bfa0e2736606eaddfe9.js"></script><link href="/css/App.4aff5598f0220c63b019.css" rel="stylesheet"></head><body><div id="root"></div></body></html>
The api however still contains "galaxy_ng_version":
$ curl -s https://galaxy.ansible.com/api/ | jq '.galaxy_ng_version'
"4.10.0dev"
Update testinfra to match the current HTML.
Change-Id: I55431311ef742efdd4aa4304692e5096e1bb2895
The curl manpage explains that port isn't optional:
--resolve <[+]host:port:addr[,addr]...>
Provide a custom address for a specific host and port pair. Us‐
ing this, you can make the curl requests(s) use a specified ad‐
dress and prevent the otherwise normally resolved address to be
used. Consider it a sort of /etc/hosts alternative provided on
the command line. The port number should be the number used for
the specific protocol the host will be used for. It means you
need several entries if you want to provide address for the same
host but different ports.
Change-Id: I40117768bbc149678a69905a8f6ecd3519301ce1
Previously we checked that "Ansible Galaxy" shows up in the html result
requesting the root of the Galaxy proxy. This now fails and looking at
the results of the fetch the title of the page is "Galaxy NG". Update
our test to check for "Galaxy NG" instead.
Additionally our content checks of actual collections are affected by an
api bump from v2 to v3. Among other things this appears to be a
completely new implementation that does not have backward compatible
support for v2 and may require authentication to use. I've commented out
our old test content for the content checks and someone will need to fix
this later.
Change-Id: I6b17eea82ac95200ba5069de74e9a7dc30d6fed8
The mix of <Location> and ProxyPass [path] <target> lead to some issue.
This patch corrects them and makes the config more consistent.
Until now, the last URI was actually an error page from the main galaxy
website. With this change, we now hit the S3 bucket as we should,
allowing ansible-galaxy to download the archive, validate its checksum,
and install the intended collection/role.
This patch was fully tested locally using the httpd container image, a
minimal configuration adding only the needed modules and the
ansible-galaxy vhost/proxy, and running ansible-galaxy directly.
In addition, this patch also makes a better testing of the proxy, using
cURL until we actually download the file.
Since ansible galaxy will provide a file under any condition, we also
assert the downloaded file is really what it should be - a plain
archive. If it's a miss on S3 side, it would be a JSON. And if we get an
ansible galaxy answer, that would be an HTML file.
The following commands were used:
Run the container:
podman run --rm --security-opt label=disable \
-v ./httpd.conf:/usr/local/apache2/conf/httpd.conf:ro \
-p 8080:8080 httpd:2.4
Run ansible-galaxy while ensuring we don't rely on its own internal
cache:
rm -rf operator ~/.ansible/galaxy_cache/api.json
ansible-galaxy collection download -vvvvvvv \
-s http://localhost:8080/ -p ./operator tripleo.operator
Then, the following URI were shown in the ansible-galaxy log:
http://localhost:8080/http://localhost:8080/apihttp://localhost:8080/api/v2/collections/tripleo/operator/http://localhost:8080/api/v2/collections/tripleo/operator/versions/?page_size=100http://localhost:8080/api/v2/collections/tripleo/operator/versions/0.9.0/
Then, the actual download:
http://localhost:8080/download/tripleo-operator-0.9.0.tar.gz
Then the checksum validation, and eventually it ended with:
Collection 'tripleo.operator:0.9.0' was downloaded successfully
Change-Id: Ibfe846b59bf987df3f533802cb329e15ce83500b
ansible-galaxy CLI makes multiple calls to the remote server, with
various API endpoint, and expects JSON containing fully qualified URI
(scheme://host/path), meaning we must inspect the different files and
ensure we're rewriting the content so that it points to the proxy all
the time.
Also, the remote galaxy.ansible.com has some redirects with absolute
paths, breaking for some reason the ProxyPassReverse - this is why we
get yet a new pair of dedicated ports for this proxy (TLS/non-TLS).
Then, there's the protocol issue: since mod_substitute is apparently
unable to take httpd variables such as the REQUEST_SCHEME, we have to
use some If statement in order to ensure we're passing the correct
scheme, being http or https. Note that ansible-galaxy doesn't understand
the "//host/path".
This patch also adds some more tests in order to ensure the API answers
as expected through the proxy.
Change-Id: Icf6f5c83554b51854fabde6e4cc2d646d120c0e9
This is rather different to all our existing testing, probably because
it was just written earlier.
Convert this all to curl calls like everything else. Don't use direct
IP addresses, but use the hostnames. Drop the --insecure flags as the
certificates cover the hostnames now. Also drop the separate ipv6
testing as some hosts don't have ipv6; what we are really interested
in is if the apache config is responding correctly, not the test node
networking setup.
Change-Id: I489055e89bfd8dd05487985dd408767b870c3980
Ansible Galaxy indexes tarballs of Ansible roles and collections at
a central site, which in turn points to a dedicated Amazon S3
subdomain. The tools which consume it support overriding the default
Galaxy URL with any arbitrary one, so should be able to take
advantage of this in CI jobs.
Change-Id: Ib5664e5588f7237a19a2cdb6eec3109452e8a107
Docker has long planned to turn this off and it appears that they have
done so. Planning details can be found at:
https://www.docker.com/blog/registry-v1-api-deprecation/
Removing this simplifies our configs as well as testing. Do this as part
of good hygiene.
Change-Id: I11281167a87ba30b4ebaa88792032aec1af046c1
We need to tell apache to listen on the ports used by the Quay Registry
Mirror. Without this we aren't actually able to provide connections to
this vhost.
Add testing to ensure this is working in a simple manner.
Change-Id: I28bdb7aeb9c3252c6319658acaa530a7d7c25a72
This was missed in an earlier change where we enabled these vhosts.
Testing worked because testing was communicating to localhost and not
the public ip address.
This has been addressed as well.
Change-Id: I2d91aea466f1b587780a452cfe8e1396515930ed
Previously we had enabled SSL on our main vhost for the mirrors. Do
similar for all of the proxy cache vhosts for docker and other external
resources.
As part of this change we improve the testing to ensure that the new
vhosts are working as expected. One testing specific change to note is
the testinfra node names did not match our existing system-config-run
job nodenames. This has been corrected.
Additionally RHRegistryMirror and QuayMirror may not be working and
fixing those is left as a followup.
Change-Id: I9dbbd4080c3a2cce4acc39d63244f7a645503553
This impelements mirrors to live in the opendev.org namespace. The
implementation is Ansible native for deployment on a Bionic node.
The hostname prefix remains the same (mirrorXX.region.provider.) but
the groups.yaml splits the opendev.org mirrors into a separate group.
The matches in the puppet group are also updated so to not run puppet
on the hosts.
The kerberos and openafs client parts do not need any updating and
works on the Bionic host.
The hosts are setup to provision certificates for themselves from
letsencrypt. Note we've added a new handler for mirror nodes to use
that restarts apache on certificate issue/renewal.
The new "mirror" role is a port of the existing puppet mirror.pp. It
installs apache, sets up some modules, makes some symlinks, sets up a
cleanup cron job and installs the apache vhost configuration.
The vhost configuration is also ported from the extant puppet. It is
simplified somewhat; but the biggest change is that we have extracted
the main port 80 configuration into a macro which is applied to both
port 80 and 443; i.e. the host will have SSL support. The other ports
are left alone for now, but can be updated in due course.
Thus we should be able to CNAME the existing mirrors to new nodes, and
any existing http access can continue. We can update our mirror setup
scripts to point to https resources as appropriate.
Change-Id: Iec576d631dd5b02f6b9fb445ee600be060f9cf1e
