We must have missed this, I noticed when it didn't run on the gate job
for I949c40e9046008d4f442b322a267ce0c967a99dc
Change-Id: I62c5c0f262d9bd53580367dc9f1ad00fe7b6f6f2
This adds upgrade testing from our current Gerrit version (3.5) to the
likely future version of our next upgrade (3.6).
To do so we have to refactor the gerrit testing becase the 3.5 to 3.6
upgrade requires we run a command against 3.5. The previous upgrade
system assumed the old version could be left alone and jumped straight
into the upgrade finally testing the end state. Now we have split up the
gerrit bootstrapping and gerrit testing so that normal gerrit testing
and upgrade testing can run these different tasks at different points in
the gerrit deployment process.
Now the upgrade tests use the bootstrapping playbook to create users,
projects, and changes on the old version of gerrit before running the
copy-approvals command. Then after the upgrade we run the test assertion
portion of the job.
Change-Id: Id58b27e6f717f794a8ef7a048eec7fbb3bc52af6
This adds Gerrit 3.6 image build jobs as well as CI testing for this
version of Gerrit. Once we've got images that build and function
generally we'll reenable the upgrade job and work through that.
Change-Id: I494a21911a2279228e57ff8d2b731b06a1573438
This removes our Gerrit 3.4 image builds as well as testing. We should
land this after an appropriate amount of time has passed since the 3.5
upgrade that we are unlikely to revert.
Depends-On: https://review.opendev.org/c/openstack/project-config/+/847057
Change-Id: Iefa7cc1157311f0239794b15bea7c93f0c625a93
We've upgraded to Gerrit 3.5 so now need to wait for the 3.5 image to
promote rather than the 3.4 image when deploying Gerrit.
Change-Id: Ic3a4d578aea955aeee51f4cac7f4c95de931a94b
Now that we've cleaned up the old unused images we can look forward to
new Python. Add Python 3.10 base images based on Bullseye.
As part of this process we update the default var values in our
Dockerfiles to set Bullseye and Python3.10 as our defaults as these
should be valid for some time. We also tidy up some yaml anchor names to
make future copy and paste for new versions of images easier to perform
text replacement on.
Change-Id: I4943a9178334c4bdf10ee5601e39004d6783b34c
Everything is running on 3.8 or newer which should allow us to remove
the 3.7 images. This reduces the total set before we add python3.10
images and acts as good cleanup.
Change-Id: I2cc02fd681485f35a1b0bf1c089a12a4c5438df3
We've moved onto bullseye for just about everything at this point. It is
possible there are stragglers and the removal of these jobs should flag
them if their dependencies and requirements are set properly. Otherwise
they'll continue to pull the historical builds on docker hub. Either way
we'll either shake them out or they will continue until they can move to
bullseye.
We remove these in preparation for adding python3.10 images which don't
make sense for buster and our total image catalog is getting large
enough that successfully building and promoting this entire set is
getting problematic. A bit of spring cleaning on what we can commit to
before we commit to some new stuff.
Depends-On: https://review.opendev.org/c/opendev/gear/+/838402
Depends-On: https://review.opendev.org/c/opendev/storyboard/+/838403
Change-Id: I58c4d314ca4f4be3f1e17ec267a4c324cabf0c2a
We don't use buster images anymore for anything. Update our dependency
on buster indicators to up to date and current bullseye dependencies.
Change-Id: I8da237559e074ae3d44be1dde8ffb7da89104d4f
This triggers the test job on changes to any gitea.* roles, including
gitea-lb which wasn't included before.
It also removes the letescrypt job as a soft dependency from the lb
jobs since that is not strictly necessary.
Change-Id: Ie5bcd4d8215bb14d939dddf3e20d3173ccc0acdc
We removed the promote jobs for Gerrit 3.3 images but left them in place
as infra-prod-service-review dependencies. Fix that by updating the
infra prod job dependencies to the job for Gerrit 3.4 image promotion.
Change-Id: If2277799db91ea61aaffafb600f403531a0fb562
This reenables Gerrit upgrade testing but tests the 3.4 to 3.5 upgrade
now. Note this may need some work to get happy once we have 3.5 images
which is why we've split it out into a separate change.
Change-Id: Ibbbd3f98ac2df8d99d4ffda57df59f4a47da3cd3
This will build gerrit 3.5 images and run it through our standard Gerrit
testing. Upgrade testing from 3.4 to 3.5 to follow in followup changes.
Change-Id: I76d0389d1455e62b242aad1926b3a09830301801
We've upgraded to 3.4 and don't appear to be reverting. Remove the 3.3
images as they are no longer needed.
Note we comment out the review upgrade testing jobs until we have 3.5
images building.
Depends-On: https://review.opendev.org/c/openstack/project-config/+/827562
Change-Id: I0e3cb81b790ab06c690ed0245526e4f47911c584
We dropped making our own grafana container with with
If0d584f848f213aeea385885e3decfaee6303de5, so we don't need this job
any more.
Change-Id: Ide212f25cda6d25e5cc31b0e8d2a65f3759bafdd
Instead of building a local grafana image with grafyaml installed,
use the plain upstream grafana image along with the newly created
separate opendev grafyaml image to run the dashboards.
Depends-On: https://review.opendev.org/780119
Change-Id: If0d584f848f213aeea385885e3decfaee6303de5
This is general spring cleaning that we are going to try and do for our
images now that bullseye is out.
Change-Id: Iad8f5b76896b88a6aafbfba0c38d0749b9d5c88f
This is a typo from the job shuffle in
I8f6150ec2f696933c93560c11fed0fd16b11bf65 -- this should be a soft
dependency.
It is currently causing periodic jobs to fail
Change-Id: Ia420e74a1d64b12b63b1697e61992c46119451dc
This used to be called "bridge", but was then renamed with
Ia7c8dd0e32b2c4aaa674061037be5ab66d9a3581 to install-ansible to be
clearer.
It is true that this is installing Ansible, but as part of our
reworking for parallel jobs this is the also the synchronisation point
where we should be deploying the system-config code to run for the
buildset.
Thus naming this "boostrap-bridge" should hopefully be clearer again
about what's going on.
I've added a note to the job calling out it's difference to the
infra-prod-service-bridge job to hopefully also avoid some of the
inital confusion.
Change-Id: I4db1c883f237de5986edb4dc4c64860390cc8e22
This adds a keycloak server so we can start experimenting with it.
It's based on the docker-compose file Matthieu made for Zuul
(see https://review.opendev.org/819745 )
We should be able to configure a realm and federate with openstackid
and other providers as described in the opendev auth spec. However,
I am unable to test federation with openstackid due its inability to
configure an oauth app at "localhost". Therefore, we will need an
actual deployed system to test it. This should allow us to do so.
It will also allow use to connect realms to the newly available
Zuul admin api on opendev.
It should be possible to configure the realm the way we want, then
export its configuration into a JSON file and then have our playbooks
or the docker-compose file import it. That would allow us to drive
change to the configuration of the system through code review. Because
of the above limitation with openstackid, I think we should regard the
current implementation as experimental. Once we have a realm
configuration that we like (which we will create using the GUI), we
can chose to either continue to maintain the config with the GUI and
appropriate file backups, or switch to a gitops model based on an
export.
My understanding is that all the data (realms configuration and session)
are kept in an H2 database. This is probably sufficient for now and even
production use with Zuul, but we should probably switch to mariadb before
any heavy (eg gerrit, etc) production use.
This is a partial implementation of https://docs.opendev.org/opendev/infra-specs/latest/specs/central-auth.html
We can re-deploy with a new domain when it exists.
Change-Id: I2e069b1b220dbd3e0a5754ac094c2b296c141753
Co-Authored-By: Matthieu Huin <mhuin@redhat.com>
Mixed up with gitea-lb naming.
Fixes I19db98fcec5715c33b62c9c9ba5234fd55700fd8
Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I91d077102904a2144d12bc60eb7341f1065473b4
This was introduced with I19db98fcec5715c33b62c9c9ba5234fd55700fd8
opendev-infra-prod-setup-src is the abstract parent job, we should be
using infra-prod-setup-src.
Change-Id: I7fdefe7ce60ab248f9a90b6be363eefc826f8e1f
The current opendev-infra-prod-base job sets up the executor to log
into bridge AND copies in Zuul's checkout of system-config to
/home/zuul/src.
This presents an issue for parallel operation, as every production job
is cloning system-config ontop of each other.
Since they all operate in the same buildset, we only need to clone
system-config from Zuul once, and then all jobs can share that repo.
This adds a new job "infra-prod-setup-src" which does this. It is a
dependency of the base job so should run first.
All other jobs now inhert from opendev-infra-prod-setup-keys, which
only sets up the executor for logging into bridge.
Change-Id: I19db98fcec5715c33b62c9c9ba5234fd55700fd8
Depends-On: https://review.opendev.org/c/opendev/base-jobs/+/807807
Having two groups here was confusing. We seem to use the review group
for most ansible stuff so we prefer that one. We move contents of the
gerrit group_vars into the review group_vars and then clean up the use
of the old group vars file.
Change-Id: I7fa7467f703f5cec075e8e60472868c60ac031f7
Previously we had a test specific group vars file for the review Ansible
group. This provided junk secrets to our test installations of Gerrit
then we relied on the review02.opendev.org production host vars file to
set values that are public.
Unfortunately, this meant we were using the production heapLimit value
which is far too large for our test instances leading to the occasionaly
failure:
There is insufficient memory for the Java Runtime Environment to continue.
Native memory allocation (mmap) failed to map 9596567552 bytes for committing reserved memory.
We cannot set the heapLimit in the group var file because the hostvar
file overrides those values. To fix this we need to replace the test
specific group var contents with a test specific host var file instead.
To avoid repeating ourselves we also create a new review.yaml group_vars
file to capture common settings between testing and prod. Note we should
look at combining this new file with the gerrit.yaml group_vars.
On the testing side of things we set the heapLimit to 6GB, we change the
serverid value to prevent any unexpected notedb confusion, and we remove
replication config.
Change-Id: Id8ec5cae967cc38acf79ecf18d3a0faac3a9c4b3
This shifts our Gerrit upgrade testing ahead to testing 3.3 to 3.4
upgrades as we have upgraded to 3.3 at this point.
Change-Id: Ibb45113dd50f294a2692c65f19f63f83c96a3c11
This bumps the gerrit image up to our 3.3 image. Followup changes will
shift upgrade testing to test 3.3 to 3.4 upgrades, clean up no longer
needed 3.2 images, and start building 3.4 images.
Change-Id: Id0f544846946d4c50737a54ceb909a0a686a594e
This uses the opendev assets bundle image created with
I3166679bde6d771276289b9d32e7e4407957b2f8.
The mount options require using BuildKit, hence the Dockerfile update.
Otherwise conceptually it's fairly simple; copy in the files from the
opendevorg/assets image rather than the file-system.
Change-Id: I36bdc76471eec5380a676ebcdd885a88d3985976
Move some common assets into a top-level assets/ directory. Services
can reference these assets via
https://opendev.org/opendev/system-config/raw/branch/master/assets/<file>
in <img> tags, etc.
Some services want to embed these into their images, but we wish to
only keep one canonical copy. For this, add a Dockerfile and jobs
that creates a simple bundle of assets in opendevorg/assets. This can
be referenced in other builds; the new BuildKit bind-mount is
particularly useful for this
(c.f. I36bdc76471eec5380a676ebcdd885a88d3985976).
Change-Id: I3931566eb86a0618705d276445fa0a5f659692ea
We create (a currently test only) playbook that upgrades zuul. This job
then runs through project creation and renaming and testinfra testing on
the upgraded gerrit version.
Future improvements should consider loading state on the old gerrit
install before we upgrade that can be asserted as well.
Change-Id: I364037232cf0e6f3fa150f4dbb736ef27d1be3f8
Update the file matchers to actually match the current set of puppet
things. This ensure the deploy job runs when we want it and we can catch
up daily instead of hourly.
Previously a number of the matchers didn't actually match the puppet
things because the path prefix was wrong or works were in different
orders for the dir names.
Change-Id: I3510da81d942cf6fb7da998b8a73b0a566ea7411
This is being done beacuse we don't make many changes to the
zuul-preview service but it runs in the hourly buildset starving deploy
runs. Since this doesn't change much we can move it to the daily run
instead.
If we need to update it we can run the playbook manually or land a
change to trigger it.
Change-Id: I89d2c712fcfd18bd4f694b2c90067295253b8836
