17437 Commits

Author SHA1 Message Date
Zuul
f80175b66a Merge "letsencrypt : don't use staging in the gate" 2021-10-06 12:59:58 +00:00
Ian Wienand
547a4578bd letsencrypt : don't use staging in the gate
Currently we connect to the LE staging environment with acme.sh during
CI to get the DNS-01 tokens (but we never follow-through and actually
generate the certificate, as we have nowhere to publish the tokens).
We've known for a while that LE staging isn't really meant to be used
by CI like this, and recent instability has made the issue pronounced.

This modifies the driver script to generate fake tokens which work to
ensure all the DNS processing, etc. is happening correctly.

I have put this behind a flag so the letsencrypt job still does this
however.  I think it is worth this job actually calling acme.sh to
validate this path; this shouldn't be required too often.

Change-Id: I7c0b471a0661aa311aaa861fd2a0d47b07e45a72
2021-10-06 15:34:21 +11:00
Ian Wienand
f673eda549 gerrit: add its actions.config file
This file used to be managed by puppet-gerrit and it seems we missed
converting it to Ansible.  Add it with the contents from the server.

Change-Id: I10a10166446941d2676ae9181fc74b5a1408c5ed
2021-10-05 08:36:25 +11:00
Jeremy Stanley
c80c6eeda9 Revert "Switch Fedora mirror to mirrors.mit.edu"
This reverts commit aa5623982fbecb4958ee90ab7ddc54d6d0d49747.

The MIT mirror seems to now be missing Fedora 34 indices, but the
one we were using before at UH looks just fine now.

Change-Id: I59649ea93cc1ce13715096dcd0b8f828ce6b6724
2021-09-30 03:09:50 +00:00
Clark Boylan
c9d6b4d248 Use fullchain.cer on graphite for nginx
Nginx doesn't seem to support explcit intermedate cert chains [0] and we
need to supply all of the certs together in a single file. Thankfully
acme.sh does this and calls it the fullchain.cer file. Use that in the
nginx config for graphite to fix issues with ssl verification to this
service.

[0] http://nginx.org/en/docs/http/configuring_https_servers.html#chains

Change-Id: I318fb92a30c1593c2a2e4cb37496b16f17472f1d
2021-09-29 15:18:10 -07:00
Zuul
bb0f135b5b Merge "GC/pack gitea repos every other day" 2021-09-28 21:35:57 +00:00
Clark Boylan
7df09ecef5 Properly copy gerrit static files
Dockerfile's COPY directive only copies the contents of a directory when
src is a directory. It does not copy the directory itself. This meant
the copy we were using to copy static files put them in /var/gerrit and
not /var/gerrit/static where we need them to be.

Update the Dockerfile to copy to /var/gerrit/static/ to fix this and add
some resource fetching tests to ensure they are served correctly.

Change-Id: I3bb4c06f3d7a57dcfccbbdb27cb8405586949949
2021-09-27 15:04:15 -07:00
Zuul
bcb437237a Merge "gerrit: host logo in static files" 2021-09-27 16:54:19 +00:00
Zuul
64d0939da2 Merge "gerrit: copy static files directly into container image" 2021-09-27 16:54:16 +00:00
Zuul
dfbd6bfa93 Merge "Upgrade gitea to 1.14.7" 2021-09-27 16:54:13 +00:00
Zuul
7e030a32df Merge "Use Apache to serve a local OpenDev logo on paste" 2021-09-21 23:28:04 +00:00
Jeremy Stanley
1a227489e2 Use Apache to serve a local OpenDev logo on paste
In order to avoid unnecessary browser requests to other sites,
install a copy of the OpenDev logo on the Lodgeit server and serve
it from there rather than pointing at one served from Gitea.

Change-Id: I4c3678a1de8ca4a41cd0c64aab71b2e0e25373af
2021-09-21 20:28:29 +00:00
Clark Boylan
eb27f5e5b7 Upgrade gitea to 1.14.7
This has a number of bugfixes and seems like a good intermediate step
before we upgrade to 1.15.x.

Change-Id: If30693a849e34a4eed6fdec09b75c438f815621d
2021-09-21 11:26:39 -07:00
Clark Boylan
522afa85cc GC/pack gitea repos every other day
Previously we were doing this weekly. Gerrit does this daily. "Split"
the difference and do gitea every other day.

We have noticed that replication to gitea can be slow at times. One idea
is that the less packed repos on the gitea side may make negotiating the
updates slower. Pack more often to see if this helps.

Change-Id: I8961007dce3e448bfdbf1c5f3e8dfc5ec8eb82fb
2021-09-21 08:32:15 -07:00
Zuul
fc75255ea9 Merge "gerrit: copy theme plugin from plugins/" 2021-09-21 15:13:09 +00:00
Zuul
02705bf73a Merge "lodgeit: use logo from system-config assets" 2021-09-21 14:28:18 +00:00
Zuul
3791aa78c7 Merge "docs: use logo from assets" 2021-09-20 19:42:57 +00:00
Zuul
856ee196f5 Merge "gitea: cleanup logo assets" 2021-09-20 19:42:54 +00:00
Zuul
57d2e67a5c Merge "Run daily backups of nodepool zk image data" 2021-09-20 17:34:28 +00:00
Ian Wienand
f19e41e893 gerrit: host logo in static files
Instead of using the opendev.org/... logo file, host a copy from
gerrit's static location and use that.  This isolates us from changes
to the way gitea serves its static assets.

Change-Id: I8ffb47e636a59e5ecc3919cc7a16d93de3eae08d
2021-09-17 12:35:12 +10:00
Ian Wienand
227839e8ad gerrit: copy static files directly into container image
Copy static files directly into the container image instead of
managing them dynamically with Ansible.

Change-Id: I0ebe40ad2a97e87b00137af7c93a3ffa84929a2e
2021-09-17 12:35:12 +10:00
Ian Wienand
e8c456926c lodgeit: use logo from system-config assets
This currently uses a file served from gitea's staic assets; to
isolate us from changes to gitea's file layout switch this to use the
canonical file directly from system-config/assets.

Change-Id: Ibf67040af2b0a18261621a120ee26c78020e3ace
2021-09-17 12:35:12 +10:00
Ian Wienand
791d09be46 gerrit: copy theme plugin from plugins/
This is a minor refactor to make a follow-on that copies static files
more logical.

Change-Id: Id31106a875926dd4eca31972b1fc52a1cd0b67bd
2021-09-17 12:35:12 +10:00
Ian Wienand
1d5f5a7657 docs: use logo from assets
Symlink the docs logo to the canonical assets location.  It looks like
it does the sensible thing and de-references the source symlink when
building, as doc/build/html/_static/opendev.svg ends up as the actual
file, not a symlink.

Change-Id: I4409c8e20601bdcb9e387d028b5df13c90d1ffa0
2021-09-17 12:35:12 +10:00
Ian Wienand
8f8100ed28 gitea: cleanup logo assets
I can not find anywhere we use the logo-*.png files currently in the
gitea public directory.  Remove these as they're all trivially
re-created via the SVG if we find we want them.

Similarly I don't see anywhere we use "opendev-icon.svg"; it is the
same as "logo.svg".  This is removed.

Move the logo with text "opendev.svg" into the assets bundle (only
user of this I can find is paste.opendev.org).

Also move in the favicon.<png/svg> files

Note all these are copied back into the gitea container at the same
location so nothing will break.  However we wish to have the
identified external users not rely on files served as gitea assets as
the new version of gitea will move them.

Change-Id: I4f6c64b4042a3f0a17ce4ee59ee8bd0d61648bcc
2021-09-17 12:35:07 +10:00
Clark Boylan
a1cf5b3f6f Run daily backups of nodepool zk image data
This does local backups of the nodepool zk image image data to
/var/log/nodepool on the nodepool-builders. These hosts don't get
offsite backups but we run mutliple redundant servers. This data isn't
critical as we can start from scratch, but may be useful if we don't
want to go through all that trouble.

Change-Id: I7d150df9c0d9566ef2d32167cea535e29822cfa2
2021-09-16 14:12:08 -07:00
Clark Boylan
f1bcb6a586 Set a gerrit replication timeout of 15 minutes
We are seeing that replication tasks occasionally sit around forever and
have had to take manual intervention. One theory is that this is related
to networking between the gerrit server and the gitea servers. We don't
set maxRetries which means replication should be retried infinitely
which means if we hit the timeout we should try again. 15 minutes was
sort of arbitrarily chosen as ~twice the time it takes to clone a large
repo like nova.

Change-Id: Iec2536ad149a2e625a1f0107b9fcee3079493607
2021-09-15 16:25:04 -07:00
Clark Boylan
6c115cf29e Add support for Ubuntu Focal to our mailman ansible
This switch testing of lists.openstack.org to Focal and we make a CGI
env var update to accomodate newer mailman.

Specifically newer mailman's CGI scripts filter env vars that it will
pass through. We were setting MAILMAN_SITE_DIR to vhost our mailman
installs with apache2, but that doesn't pass the filter and is removed.
HOST is passed through so we update our scripts, apache vhost configs,
exim, and init scripts to use the HOST env var instead.

Change-Id: I5c8c70c219669e37b7b75a61001a2b7f7bb0bb6c
2021-09-13 09:10:00 -07:00
Zuul
4db138edfd Merge "testinfra: refactor screenshot taking" 2021-09-07 17:05:31 +00:00
Zuul
7a90a92ebb Merge "gitea: add some screenshots to testing" 2021-09-07 16:57:45 +00:00
Zuul
669300d7e2 Merge "gitea: use assets bundle" 2021-09-07 16:57:41 +00:00
Zuul
7a9c9475a7 Merge "Add assets and a related docker image/bundle" 2021-09-07 15:47:23 +00:00
Ian Wienand
9ba398dee9 testinfra: refactor screenshot taking
Reduce the screenshots to a single utility function to avoid copying a
lot of boilerplate.

Change-Id: Iad1c7afa4e9ea9a4ddaca5e62751795e60bc2980
2021-09-07 12:54:26 +10:00
Ian Wienand
1dde7628e8 gitea: add some screenshots to testing
Change-Id: Id13fdd8ffbca1b0cd19858419d68f012e33f3ba8
2021-09-07 08:59:46 +10:00
Ian Wienand
e772abaf96 gitea: use assets bundle
This uses the opendev assets bundle image created with
I3166679bde6d771276289b9d32e7e4407957b2f8.

The mount options require using BuildKit, hence the Dockerfile update.

Otherwise conceptually it's fairly simple; copy in the files from the
opendevorg/assets image rather than the file-system.

Change-Id: I36bdc76471eec5380a676ebcdd885a88d3985976
2021-09-06 15:07:36 +10:00
Zuul
300982c6c1 Merge "Accomodate zuul's new key management system" 2021-09-01 15:56:40 +00:00
Zuul
da558e10c3 Merge "Introduce iweb cloud configs" 2021-08-31 23:38:46 +00:00
Mathieu Gagné
d518f8c02c Introduce iweb cloud configs
INAP mtl01 region is now owned by iWeb. This updates the cloud launcher
to use the new name and instructs the mirror in this cloud to provision
ssl certs for the old inap and new iweb names as well as updating
clouds.yaml files.

Change-Id: I1256a2e24df1c79dea06716ae4dfbcfe119c13f8
2021-08-31 13:42:22 -07:00
Ian Wienand
25cdc97950 Add assets and a related docker image/bundle
Move some common assets into a top-level assets/ directory.  Services
can reference these assets via

 https://opendev.org/opendev/system-config/raw/branch/master/assets/<file>

in <img> tags, etc.

Some services want to embed these into their images, but we wish to
only keep one canonical copy.  For this, add a Dockerfile and jobs
that creates a simple bundle of assets in opendevorg/assets.  This can
be referenced in other builds; the new BuildKit bind-mount is
particularly useful for this
(c.f. I36bdc76471eec5380a676ebcdd885a88d3985976).

Change-Id: I3931566eb86a0618705d276445fa0a5f659692ea
2021-09-01 06:15:43 +10:00
Jeremy Stanley
2fbf6d9e7a Stop managing OpenStackID servers
The Open Infrastructure Foundation's developers who maintain the
OpenStackID software are taking over management of the site itself,
and have deployed it on new servers. DNS records have already been
updated to the new IP address, so it's time to clean up our end in
preparation for deleting the old servers we've been running.

OpenStackID is still used by some services we run, like RefStack and
Zanata, and we're still hosting the OpenStackID Git repository and
documentation, so this does not get rid of all references to it.

Change-Id: I1d625d5204f1e9e3a85ba9605465f6ebb9433021
2021-08-31 19:53:13 +00:00
Zuul
861f4698aa Merge "Switch robots.txt test on gitea to proxy port" 2021-08-30 02:04:50 +00:00
Monty Taylor
8dbf0a3d82 Produce both buster and bullseye container images
This will allow us to roll out consumption forward in a
methodical manner.

This reverts commit 45caec4d43900bc66fb0b8c219c6dcc3180ca8aa.

Note the weird ARG definitions are there for a reason:
https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact

Change-Id: I81174ac035164695f1c27c9662f25335b78c2e64
2021-08-28 16:46:29 -07:00
Zuul
0f3b1f649b Merge "Pin base and builder images to buster" 2021-08-27 17:11:01 +00:00
Zuul
6969f4a033 Merge "Test lists.kc.io on focal" 2021-08-27 16:00:48 +00:00
James E. Blair
45caec4d43 Pin base and builder images to buster
There's some more work before our consumers can switch to bullseye.
To make this process more tractable, revert the recent backport
addition, and specify that we want bullseye images from upstream.

That gets us back to where we were at the start of this.  Next,
we can start building 2x images of python-base/builder and tag
them with bullseye or buster.  Then the consumers can specify
which tags, then start switching.

Revert "Add backports repos to base and builder images"

This reverts commit b217e38904da3ccab6eb96251376f1635ee55d21.

Revert "Update matrix-eavesdrop for bullseye"

This reverts commit fc38c6975367c09d003474ebd7bfefc465459a06.

Change-Id: Id21681342fe5268296128c1b09436a80c46e3169
2021-08-27 08:03:38 -07:00
Zuul
84dca1ecf4 Merge "Update OpenSUSE mirror source" 2021-08-26 22:57:58 +00:00
Zuul
a6beb03ceb Merge "Update matrix-eavesdrop for bullseye" 2021-08-26 22:34:09 +00:00
Zuul
aaeebdda18 Merge "Add backports repos to base and builder images" 2021-08-26 20:37:44 +00:00
Monty Taylor
fc38c69753 Update matrix-eavesdrop for bullseye
Bullseye means we don't need the backports repos. Also, the upstream
images have bullseye-backports in them already now.

Depends-On: https://review.opendev.org/c/opendev/system-config/+/800318
Change-Id: I3813068c21d06d9b182fe81efcf2e636b2170c4a
2021-08-26 15:11:37 -05:00
Monty Taylor
b217e38904 Add backports repos to base and builder images
These don't install anything by default, but allow people
to easily opt-in to a backport package if they need to without
lots of extra lines of boilerplate.

The base python image is on bullseye now instead of buster. That
means the libffi version is 7 not 6.

Change-Id: I0e0c2669d838fb622422f696f73e96e409157270
2021-08-26 11:58:59 -05:00