28 Commits

Author SHA1 Message Date
Clark Boylan
999edcc88b Remove melody
We don't need this plugin right now

Change-Id: I7b2f0d831579076d890ef8dd3bbe6e14fa1371bc
2021-12-10 10:00:41 -08:00
Ian Wienand
6e232716e7 gerrit: update theme to javascript plugin
Gerrit 3.4 deprecates HTML-based plugins, so the old theme doesn't
work.  I have reworked this into a javascript plugin.

This should look the same, although I've achieved things in different
ways.

This doesn't register light and dark variants; since
background-primary-color is white, by setting the
header-background-color to this we get white behind the header bar,
and it correctly switches to the default black(ish) when in dark mode
(currently its seems the header doesn't obey dark mode, so this is an
improvement).

I'm not sure what's going on with the extant header-border-image which
is a linear gradient all of the same color.  I modified this down to
1px (same as default) and made it fade in-and-out of the logo colour,
just for fun.

Change-Id: Ia2e32731c1cfe97639de2ec0e7660c7ed583e045
2021-11-04 18:45:18 +11:00
Wes Wilson
bdb6e7b8f8 Update ICLA to reference OpenInfra
Change-Id: Ifc9feaefcd64bb2e6ca060b5bd3aac5ed6345f9c
2021-10-07 13:17:12 -07:00
Ian Wienand
f19e41e893 gerrit: host logo in static files
Instead of using the opendev.org/... logo file, host a copy from
gerrit's static location and use that.  This isolates us from changes
to the way gitea serves its static assets.

Change-Id: I8ffb47e636a59e5ecc3919cc7a16d93de3eae08d
2021-09-17 12:35:12 +10:00
Ian Wienand
227839e8ad gerrit: copy static files directly into container image
Copy static files directly into the container image instead of
managing them dynamically with Ansible.

Change-Id: I0ebe40ad2a97e87b00137af7c93a3ffa84929a2e
2021-09-17 12:35:12 +10:00
Ian Wienand
791d09be46 gerrit: copy theme plugin from plugins/
This is a minor refactor to make a follow-on that copies static files
more logical.

Change-Id: Id31106a875926dd4eca31972b1fc52a1cd0b67bd
2021-09-17 12:35:12 +10:00
Zuul
9181d5198d Merge "gerrit: add mariadb_container option" 2021-06-16 23:14:48 +00:00
Ian Wienand
570ca85cd8 gerrit: add mariadb_container option
This adds a local mariadb container to the gerrit host to hold the
accountPatchReviewDb database.  This is inspired by a few things

 - since migration to NoteDB, there is only one table left where
   Gerrit records what files have been reviewed for a change.  This
   logically scales with the number of reviews users are doing.
   Pulling the stats on this, we can see since the NoteDB upgrade this
   went from a very busy database (~300 queries/70 commits per second)
   to barely registering one hit per second :
   https://imgur.com/a/QGJV7Fw

   Thus separating the db to an external host for performance reasons
   is not a large concern any more.

 - emperically we've done a bad job in keeping the existing hosted db
   up-to-date; it's still running mysql 5.1 and we have been hit by
   bugs such as the one referenced in-line which silently drops
   backups.

 - The other gerrit option is to use an on-disk H2 database.  This is
   certainly an option, however you need special tools to interact
   with it for migration, etc. and it's not safe to backup from files
   on disk (as opposed to mysqldump).  Upstream advice is unclear, and
   varies between H2 being a performance bottleneck to this being
   ephemeral data that users don't care about.  We know how to admin
   mariadb/mysql and this allows us to migrate and backup data, so
   seems like the best choice.

 - we have a pressing need to update the server to a new operating
   system.  Running the db alongside the gerrit instance minimises
   fiddling we have to do manging connections to and migrating the
   hosted db systems.

 - related to that, we are tending towards more provider independence
   for control-plane servers.  A hosted database product is not always
   provided, so this gives us more flexibility in moving things
   around.

 - the main concern here is memory usage.  "docker stats" reports a
   quiescent container, freshly started on a 8GB host:

    gerrit-compose_mariadb_1  67.32MiB

   After loading a copy of the production table, and then dumping it
   back to a file the same container reports:

    gerrit-compose_mariadb_1  462.6MiB

The existing remote mysql configuration path remains mostly the same.
We move the gerrit startup into a script rather than a CMD so we can
call it after a "wait for db" script in the mariadb_container case
(this is the reccommeded way to enforce ordering [1]).

Backups of the local container need different dump commands; backups
are relocated to a new file and updated.

Testing is converted to use this rather than a local H2 database.

[1] https://docs.docker.com/compose/startup-order/

Change-Id: Iec981ef3c2e38889f91e9759e66295dbfb499c2e
2021-06-16 13:57:13 +10:00
Clark Boylan
3639d7b535 Remove special x/ handling patch in gerrit
Upstream Gerrit has been fixing this up. It's in 3.2 as of
https://gerrit-review.googlesource.com/c/gerrit/+/306519

Change-Id: Ie8196c38629b4893364b969aeb6ac466cff0c93f
2021-06-08 13:07:19 -07:00
Monty Taylor
696b3671fb Symlink python3 to python for gerrit image build
Gerrit's bazel rules are looking for python which doesn't exist on our
images. Add a python symlink to python3 until
https://gerrit-review.googlesource.com/c/gerrit/+/298903 is in a release,
which seems likely to be 3.5.

Change-Id: I1c15cceac1c9bbf435ed23bed7c1e3fe868f05ff
2021-05-18 14:47:10 -05:00
Clark Boylan
469153d820 Use patch instead of sed to update Gerrit's x/ route
We modify the x/ route to ensure we can serve git repos from x/.
Previously we had been using sed which is likely to be much more fragile
than patch. Patch will detect conflicts and other errors which would be
good for us to find out about early.

Change-Id: Ic324c7777e7851a6150e4415338c4628ac710970
2021-01-28 08:51:12 -08:00
Ian Wienand
738b4ba739 gerrit: Install zuul-summary-results plugin
This installs the zuul-summary-results plugin into our gerrit
container.  testinfra is updated to take a screenshot of the plugin in
action.

Change-Id: Ie0a165cc6ffc765c03457691901a1dd41ce99d5a
2021-01-18 07:58:23 -08:00
Ian Wienand
951c2f4cde gerrit: get files from bazel build dir
bazel likes to build everything in ~/.cache and then symlink bazel-*
"convience symlinks" in the workspace/build directory.  This causes a
problem for building docker images where we run in the context of the
build directory; docker will not follow the symlinks out of build
directory.

Currently the bazelisk-build copies parts of the build to the
top-level; this means the bazelisk-build role is gerrit specific,
rather than generic as the name implies.

We modify the gerrit build step to break build output symlink and move
it into the top level of the build tree, which is the context the
docker build runs in later.  Since this is now just a normal
directory, we can copy from it at will there.

This is useful in follow-on builds where we want to start copying more
than just the release.war file from the build tree, e.g. polygerrit
plugin output.

While we're here, remove the javamelody things that were only for 2.X
series gerrit, which we don't build any more.

[1] https://docs.bazel.build/versions/master/output_directories.html

Change-Id: I00abe437925d805bd88824d653eec38fa95e4fcd
2021-01-18 07:58:23 -08:00
Ian Wienand
e80893eb3d bazelisk-build: specify targets as list
Specify bazelisk_targets as a list, and join the targets as
space-separated in the build command.  This is used in the follow-on
Ie0a165cc6ffc765c03457691901a1dd41ce99d5a.

While we are here, remove the build-gerrit.sh script that isn't used
any more, along with the step that installs it.

Also, refactor the tasks to use include_role (this is also used in the
follow on).

Change-Id: I4f3908e75cbbb7673135a2717f9e51f099a4860e
2021-01-18 07:58:23 -08:00
Ian Wienand
1858f15a8a gerrit: move plugins to common code
The "additional_plugins" variable is so different builds gerrit can
specify additional plugins specific to their version to install into
the base image.

Since we've moved to only building 3.2 and master images, a bunch of
plugins that used to be additional (because they weren't 2.XX era) are
now common.  Move them into the common plugin code in the playbook,
and leave the only one different for master, the "checks" plugin, as
separate.

Change-Id: I8966ed7b5436fbe012486dccc1028bc8cb1cf9e4
2021-01-18 07:58:23 -08:00
Paladox
d7b726140d Add an OpenDev theme into our Gerrit image
This provides an HTML-only PolyGerrit plugin consistent with our
Gitea theming, generously provided by Paladox (many thanks!).

Since we have to split some roles in the build playbook, also name
the temporary patching role to make the build console a little
easier to read.

Change-Id: I3baf17d04b2dca34fc23dcab91c00544cedf0ca6
2020-12-04 18:01:11 +00:00
Clark Boylan
70ef807b3e Update gerrit docker image to java 11
Gerrit 3.2 supports java 11 now and Gerrit 3.3 will be the last to
support java 8. Lets get ahead of things and switch to java 11.

Change-Id: I1b2f6b1bdadad10917ef5c56ce77f7d7cfc8625d
2020-12-02 18:34:07 +00:00
Clark Boylan
01dab90bc1 Handle x/ prefix projects on gerrit 3.2
Gerrit seems to handle x/ for plugin extensions in polygerrit.
Unfortunately we've got projects called x/* and that breaks cloning of
these projects. Lets just avoid that for nwo until we can do a rename.

Change-Id: Id01739725c22af9d02ac30b1653743b49a35a332
2020-11-21 13:58:34 -08:00
Clark Boylan
1008a1fc90 Build images for gerrit 3.1 and 3.2
This will allow us to test further gerrit upgrades while we sort out how
far into the gerrit releases we will be upgrading to on our next
upgrade.

Change-Id: Ic9d07b76e41ad4262cc0e2e1ff8a5d554f88239e
2020-10-06 15:37:06 -07:00
Jeremy Stanley
dd697fc313 Use ensure-nodejs in Gerrit deployment testing
The install-nodejs role in zuul-jobs has been replaced by
ensure-nodejs, so we should use the new thing if we want our tests
running again.

Change-Id: I196814b616d3b332b2c1d397097c01b5bb0d2aac
2020-05-19 19:16:26 +00:00
Monty Taylor
a8e1d1496d Build gerrit images with bazelisk
We need to use bazelisk to build gerrit so that we can properly
track bazel versions in the job. Use the roles developed for
gerrit-review to do that, then simplify the dockerfile to have
it simply copy the war into the target image.

Also add polymer-bridges.

Depends-On: https://review.opendev.org/709256
Change-Id: I7c13df51d3b8c117bcc9aab9caad59687471d622
2020-02-21 17:32:01 -06:00
Monty Taylor
36aa77937a Add jobs to build gerrit master branch
We'll use this to test the checks plugin.

We have to add jgit as a repo because it's a submodule now.

Change-Id: Ic7e9ad0265e136a9ac6b1147998f6eb5ee398180
2019-10-20 06:35:56 +09:00
Monty Taylor
9ab25e89a9 Several updates because the world is a dark place
A few things have changed and we need to fix them in one go.

Use mirror for installing docker for buildset-registry

While, we need to make this more systemic, that's hanging off of the
mirror rework. For now, since we know all of these jobs are debian
based, just set the mirror location.

Replace use of zuul cloner with git clones

You can never be a prophet in your own hometown. This is now broken
because of the git cache rework, so just replace it.

Update libjemalloc library

python:slim is based on buster now, which has libjemalloc2 not
libjemalloc1.

Remove gerrit repo remote for submodules

A recent change to the base jobs to use prepare-workspace-git
broke the gerrit image builds by actually having the origin
remote by /dev/null as intended. This breaks submodules because
for a few of them where we don't have matching stable branches
the submodule relative path behavior is actually exactly what
we want.

Since we don't care about the remote otherwise, remove the
origin remote before doing the submodule update --init so that
the submodule will clone the refs from the zuul prepared repo.

Change-Id: Ieb5b6bc8711fe971ed3445c7c267306ac4616464
2019-10-19 07:51:29 +09:00
Monty Taylor
f0a3f0cb37 Remove bazel version hack
The upstream patch has landed, so we don't need this anymore.

Change-Id: I08a6705f189b2a24b737ab4f52bb7f449879fdf1
2019-09-19 14:18:41 +02:00
Monty Taylor
072fcca06f Fix files matcher and bazel for gerrit base image
Use latest bazel

It seems 0.27 is now too old. This is what happens when I go on vacation
apparently.

Add in a hack to override the bazelversion. We'll remove this once
https://gerrit-review.googlesource.com/c/gerrit/+/237495 lands and
has been merged up.

Change-Id: Ib7a6d33ce8bf8498fd5cd09b25087dc09acb8df4
2019-09-16 21:20:18 +02:00
Monty Taylor
56ceaf1c40 Remove the extra bazel options
We had some extra bazel options that don't seem to be necessary
anymore now that we are using upstream bazel options appropriately.

Retry the build a couple of times if it goes south, inside of the
build image. This should allow re-use of the cache the second time,
and if there is a temporary error, it should pick up and move
forward.

Change-Id: I5f304acb21fd3a4d40701fc0414ae0c424c838e5
2019-08-26 11:26:19 +02:00
Monty Taylor
2a46202b9f Build gerrit images for 2.16 and 3.0 as well
Our goal is upgrading to 3.0. To do that we need to upgrade to 2.15, then
to 2.16, then to 3.0. Build all of the images so that we can do that.

2.16 and 3.0 also use bazel, so just use one copy of the Dockerfile for
all three and let zuul check out the repos to the right versions.

Depends-On: https://review.opendev.org/673147
Depends-On: https://review.opendev.org/672320
Change-Id: I35bd278e0c70c871fa44d005c60a987d1d8e3cdc
2019-07-27 11:34:42 -04:00
Monty Taylor
b1c4a6d36a Build a gerrit image
The gerrit source dir needs three plugins cloned into
the plugins dir and also a few files updated.

Depends-On: https://review.openstack.org/631007
Change-Id: I56037137d43ee1cea0a4c17e48d09102e1599ddc
2019-01-16 11:27:20 -08:00