Make image and volume list in compose file templated.
Rename the gerrit-podman directory to not be based on tool.
Make sure we run the job on changes to the playbooks.
Update the job name - it's not just review-dev anymore.
Change-Id: I0341fa95caff656a2176cc2026ec0ac8903fb24e
NOTE: We should update storyboard-dev to be driven by
letsencrypt first, otherwise we need to plumb in the
self-signed cert, which gets weird with needing to
import it for java which in this case is in the container
image, meaning we either need to bind-mount java certs in
or build it in to the image.
Change-Id: Ida9dd15ca8262925c54579660fe9c16e2b573907
Our base images have pip pre-installed from get-pip. This means
that the installation of pip and virtualenv from distro packages
in the ansible is misleading.
Update the role to match reality.
Change-Id: I500b14f9f9df00b6e0c4f152f8b4c7faa1bb94d4
To make it clear that docker hub is but one of many possible registries,
update our usage of FROM and image: lines to include docker.io in the
path.
There are a few other FROM lines for the gitea images which are handled
in a separate stack.
Change-Id: I6fafd5f659ad19de6951574afc9a6b6a4cf184df
1.10 introduces a PASSWORD_COMPLEXITY setting with a default value
of lower,upper,digit,spec - which requires passwords to have an
upper, lower, digit and special character. Our example password does
not have this, so set the PASSWORD_COMPLEXITY setting. We could
alternately leave it at the default and ensure that our passwords
meet the spec.
The sshd_config file is templated now, so we can set the listen port
via env var.
Change-Id: I6e4b595eabb9c6885d78fff1109ea9f602e89ef7
We switched Fedora to do vos release via ssh with localauth in
I56ecdb2511597197deeeadf51f50da7e02f56954 and it has been working.
Switch the rest of the update scripts. There is an increasing amount
of common code, start a common functions.sh script where we can put
this.
Change-Id: I4ba6d64a84bb66e8686901b16010352de942f303
Use the new vos_release user on the remote host to release the volume
via localauth, to avoid any timeouts.
Change-Id: I56ecdb2511597197deeeadf51f50da7e02f56954
Depends-On: https://review.opendev.org/#/c/695554/
The sudoers parser really, really, *really* doesn't like it when the
last line of data in your file lacks a trailing newline. Add one so
sudo will work again on these servers.
Change-Id: I40fbb535faf5b41cc56c56f09f248eea398df4e0
If you read the man page
# This will cause sudo to read and parse any files in the /etc/sudoers.d
# directory that do not end in '~' or contain a '.' character.
I don't know why sudo doesn't like files with a ".", but remove it
Fix the syntax in this file which has too many spaces
The theory that specifying a command means you can have nologin as
shell is debunked; change the shell to /bin/bash
root@mirror-update01:~# ssh -i ~/.ssh/id_vos_release vos_release@afs01.dfw.openstack.org vos
This account is currently not available.
Don't use shortcuts for positional parameters, suggested by jaltmann
in If70b27cb974eb8c1bafec2b7ef86d4f5cba3c4c5.
After hand applying these fixes, I can log in and run the script as
expected.
Change-Id: I058aadaa5ca5c7b8e94b275c4b8d26e1e0688ce8
I was trying to simplify things by having a restricted shell script
run by root. However, our base-setup called my bluff as we also need
to setup sshd to allow remote root logins from specific addresses.
It's looking easier to create a new user, and give it sudo permissions
to run the vos release script.
Change-Id: If70b27cb974eb8c1bafec2b7ef86d4f5cba3c4c5
I wasn't correctly sourcing the key; it has to come from hostvars as
it is in a different play on different hosts. This fixes it.
We also need to not have the base roles overwrite the authorized_keys
file each time. The key we provision can only run a limited script
that wraps "vos release".
Unfortunately our gitops falls down a bit here because we don't have
full testing for the AFS servers; put this on the todo list :) I have
run this manually for testing.
Change-Id: I0995434bde7e43082c01daa331c4b8b268d9b4bc
This is the first step in managing the opendev.org cert with LE. We
modify gitea01.opendev.org only to request the cert so that if this
breaks the other 7 giteas can continue to serve opendev.org. When we are
happy with the results we can merge the followup change to update the
other 7 giteas.
Depends-On: https://review.opendev.org/694182
Change-Id: I9587b8c2896975aa0148cc3d9b37f325a0be8970
We constantly have problems with various timeouts on the release of
our mirror volumes creating locked volumes or stuck transactions; this
then requires significant manual intervention. This has been
discussed multiple times, but this short exchange from #openafs
probably sums it up best:
Sep 11 13:32:35 <auristor> The timeout problem is due to the fact
that UV_ReleaseVolume performs multiple RPCs. vos acquires a token
from the cache manager when it starts. it has no method of acquiring
a new token if it expired during an RPC. Therefore, if the token did
expire the remaining RPCs are performed unauthenticated. Without
appropriate permissions the cleanup of the volservers, writing the
updating VL entry will fail.
Sep 11 13:33:59 <auristor> A frequent solution is to deploy a remctld
service which has access to issue vos commands as -localauth and then
use remctld ACLs to restrict the identities of the processes that are
permitted to request the volume release.
Sep 11 14:37:28 <kaduk> Yeah, the -localauth tokens are pretty key
for long-running stuff, at the moment.
Indeed remctl [1] has been written to be the kerberos-based remote
control AFS wrapper. However, it is complex to setup, uses a lot of
Perl and it is unlikely to be familiar to very many people (making the
footprint of people who can help us admin it low). Getting it wrong
seems to be a pretty good vector for remote exploits. It does not
seem to be a good fit.
However, we can take a simpler approach. We can use Ansible to setup
our afs server to allow a particular key to run a release script that
wraps the "vos release -localauth" for us. With this in place, we can
update the scripts that run on mirror-update to ssh remotely and call
this, rather than call "vos release" directly.
This implements this basic support for the remote script. A new key
will be generated on mirror-update.opendev.org and it will be allowed
to run the vos_release.sh script remotely; which filters the command
to just do "vos release -localauth".
After we have tested this, we can start using it in scripts. I think
time will tell if we need locking or other features; this seems like
the KISS place to start.
[1] https://www.eyrie.org/~eagle/software/remctl/remctl.html
Change-Id: I6c96f89c6f113362e6085febca70d58176f678e7
This runs gerrit in a container on review-dev01 using podman.
Remove an unused web_server.py file that we found from copying it
from puppet to ansible.
Change-Id: I399d3cf8471bc8063022b0db0ff81718b2ee2941
Add a purge phase to remove old releases; remove Fedora 28 and stop
mirroring. Update the atomic list while we're there.
Fedora 28 was removed with Ic0b4b065a217dcfaa8c230cda53114793e93b803
Change-Id: If713844ac90ea37e8c4db30108c45d7a59832776
This server is a replacement for the .openstack.org version, which no
longer exists.
Depends-On: https://review.opendev.org/690767
Change-Id: I0d2eeb609219ad96db39d1d59b99ae376419df0e
Ceph Nautilus is released and the official mirror
is available. This adds the Ceph Nautilus mirror
so we can sync it for Stretch and Bionic.
Based on the same change that was done when Mimic
was released [1]
[1] https://review.opendev.org/#/c/571989/
Change-Id: I9424d1f4df58acde8ea70dc16283d4de89189bae
Add bool to use_upstream_docker conditional
This is an ansible behavior change that's coming in 2.12 but is
currently spewing warnings. The warnings make the log really hard
to read, so just fix it.
Disable group name auto-renaming
If you have group names with non-python identifier characters, it
prevents you from looking it up in jinja like "groups.group-name"
so ansible auto-transforms it so you can do "groups.group_name".
This is a confusing behavior which is going away. However, ansible
is warning everyone who has groups with characters in them as it
has no idea how you might be accessing those group names. Add
a config setting to suppress the warning about -'s in group names.
Change-Id: Ib3262025799af7c3171ed0b079cb1dd969075931
Just to eliminate the possibility that a bug in this may be causing
us to erroneously delete blobs.
Change-Id: I048d9ae5ba92984c90f84f231b412050a52fcea6
This is no longer used as read access is provided to unathenticated
users with the recently added JWT support.
Depends-On: https://review.opendev.org/687422
Change-Id: I36fd28710c644be9b07d645c6b0c6092f52a7385
An upcoming change will add JWT authentication to the registry;
prepare for that by establishing a server-side secret for use
in signing the tokens.
Change-Id: Ibaa15dd0c4b0d797f01a1886186fdc021dc990fa
We're trying to get clouds.yaml into /etc/openstack/clouds.yaml.
This should accomplish that. The previous configuration was in
error and wiped out the /etc directory which made things not work
well.
Change-Id: I88e69b05f3e8c1688d24736fa775163fc25a07f0
This uses the new zuul-registry container image to run the
intermediate registry. The same authentication data and certs
are used. The new registry also writes to the same swift container,
but uses different pseudo-directories so it won't clash with the
current registry. If there are problems, we can switch back easily.
After successful use of the new registry, we can delete the old data.
Change-Id: Ib855fb99c991411293a617b9b238d79a6bfae328
The extant "logrotate_daily" varaible doesn't really do anything and
isn't used. Modify this to be able to set a range of rotation periods
or a size.
Update mirror rsync mirrors to rotate weekly, as often releases run
overnight and it's a pain to reconstruct.
Change-Id: I121dc5f4fe7f226b66d18b9ec39e7e3839be4d40
This adds mirroring of CentOS 8. It is somewhat simpler because the
architecture we're interested in are in the base repos, no need for
altarch.
The current mirror doesn't have a 8/ directory; possibly they require
their own mirroring filter updates? Use an up-to-date mirror for 8
(we can switch 7 too, but leaving alone for now).
Additionally, the altarch mirror we are using appears to have gone
offline for at least a few days. Switch to another one that is in
Texas, which should be close-ish to the DFW servers.
Change-Id: I33d95fa6b2df23fbfdb6745a3079761e228f677b
Looks like leaseweb, kernel.org and others are not properly
syncing the Suse mirrors as they are out of date for Leap15.0 and
missing files (deltainfo.xml.gz) which causes job breakage for
Leap15.0 based jobs.
Revert this to the original mirror from a year ago which is updated
and not broken
Change-Id: Id7184ee973bbabfec3f601fc9200ffac17322558
This change adds a proxy config for quay which should assist
us when gating using images provided by the publically
available registry.
Change-Id: I971705e59724e70bd9d42a6920cf4f883556f673
Signed-off-by: Kevin Carter <kecarter@redhat.com>
Kolla uses this to build hacluster images.
Direct usage is causing timeouts recently.
I changed the formatting to make it more readable
and slapped a comment note for maintainers.
Change-Id: I68d7155718c0ae0744198ca96aca1a207bab7ed6
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
As described inline, this should make our mirror pulses more robust
against timeouts.
This is probably ripe for turning into more of a library situation for
all the other "vos release" calls too. But one thing at a time ... I
think we test with this for a while to see if stability returns.
Change-Id: I041a290053e4e8ceba80785598a5945e5adcf6f1
