The "additional_plugins" variable is so different builds gerrit can
specify additional plugins specific to their version to install into
the base image.
Since we've moved to only building 3.2 and master images, a bunch of
plugins that used to be additional (because they weren't 2.XX era) are
now common. Move them into the common plugin code in the playbook,
and leave the only one different for master, the "checks" plugin, as
separate.
Change-Id: I8966ed7b5436fbe012486dccc1028bc8cb1cf9e4
By setting the auth type to DEVELOPMENT_BECOME_ANY_ACCOUNT and passing
--dev to the init process, gerrit will create an initial admin user
for us. We leverage this user to create a sample project, change,
Zuul user and sample CI result comment.
We also update testinfra to take some screenshots of gerrit and report
them back.
Change-Id: I56cda99790d3c172e10b664e57abeca10efc5566
This runs selenium from a container on a node, and exposes port 4444
so you can issue commands to it. This is used in the follow-on
I56cda99790d3c172e10b664e57abeca10efc5566 to take some screenshots of
gerrit.
Change-Id: Idcbcd9a8f33bd86b5f3e546dd563792212e0751b
We don't need to test two servers in this test; remove review-dev.
Consensus seems to be this was for testing plans that have now been
superseded.
Change-Id: Ia4db5e0748e1c82838000c9b655808c3d8b74461
The upstream mirror seems to have some dotfiles that give permissions
errors when trying to rsync. Ignore all dotfiles.
Change-Id: Ic165b27ba190667f2b6b9e17cfa3ebe2ea6b9fb3
To complete our transition to borg backups, remove bup-related bits
from backup hosts. All hosts have been backing up with borg since
Ic3adfd162fa9bedd84402e3c25b5c1bebb21f3cb.
Change-Id: Ie99f8cee9befee28bcf74bff9f9994c4b17b87ff
The executors are in a loop trying to clone nova.
They start trying to refresh
/var/lib/zuul/executor-git/opendev.org/openstack/nova, it times out
after the default 5 minutes, they delete the directory and try again
... over and over. All 12 of them doing this at once kills gerrit
trying to serve nova in an endless loop.
Manual testing has shown that it takes about 7 minutes, so 10 minutes
gives us some headroom. The repo is now ~2gb I'm guessing after the
notedb changes.
Change-Id: Ie7b4268c0db447717b827dd005bb7cad3ccfbee3
This is enabled on review-test if you want to test it out. It is
intended to speed up fetches and clones and such as you don't have to
list and interact with all of the gerrit change refs to do those common
operations.
Change-Id: I65b430548a2805cd05dc4cdbcf9354a9c18faadc
We are seeing java gc go crazy at times and aren't quite sure what is
causing it. Add jvm gc logging to the gerrit process to help us identify
what is happening.
Additionally we add SYS_PTRACE to the container capabilities so that you
can get heap dumps from the jvm. To get a heap dump you need to do
roughyl:
docker exec -u root -it gerritcontainerid jhsdb jmap --heap --pid $pid
Change-Id: Ib4a5b84fda4eca73c7971c31ee74c3232eb733e4
We were setting these values in gerrit.config but it isn't clear if
these now need to go in jgit.config instead. I've tried to clarify with
upstream maintainers as the documentation is quite confusing. While we
wait for clarification why don't we just set the values in both files to
ensure we are covered.
This converts jgit.config to a jinja2 template so that we can use the
variable number of pack files setting.
Change-Id: I70c1e6b738ed6e9fdb72d86e7cf3fb8cfecf1323
This provides an HTML-only PolyGerrit plugin consistent with our
Gitea theming, generously provided by Paladox (many thanks!).
Since we have to split some roles in the build playbook, also name
the temporary patching role to make the build console a little
easier to read.
Change-Id: I3baf17d04b2dca34fc23dcab91c00544cedf0ca6
Gerrit 3.2 supports java 11 now and Gerrit 3.3 will be the last to
support java 8. Lets get ahead of things and switch to java 11.
Change-Id: I1b2f6b1bdadad10917ef5c56ce77f7d7cfc8625d
The receive.autogc configuration apparently needs to be in its own
file, not in the general gerrit.conf. Move it to the correct
location.
https://review.opendev.org/Documentation/config-gerrit.html#jgit-receive
While we're here, correct the filename on the gerrit.config and
secure.config templates to make it clear they're jinja2 files, and
add a file mode to the replication.config where it was missing.
Change-Id: I9243bccac103c51ee435725aae482731642a37cc
For unfathomable reasons, Gerrit implements automatic GC on every
push and enables this by default but recommends in the documentation
that it be turned off. Follow their recommendation on this, since it
seems to result in additional load and we already periodically GC
all repos anyway.
Change-Id: I9a46c69b26e0a746f2aed308a28e5408e5c34ef1
We're seeing high system load and decreased performance on our
production Gerrit instance. Some research suggests this may be I/O
contention which can be relieved through better caching:
https://groups.google.com/g/repo-discuss/c/7CemrH4lVJE
According to `gerrit show-caches --show-jvm --show-threads` some of
our memory-only caches are already at their default maximums after
only a few days of operation, and one in particular
(changeid_project) has a particularly poor cache hit ratio of 24% at
the moment. Increase changeid_project from the 1024 entry default by
32x (manual tests at 8x approached 50% cache hit), increase projects
by 4x (greater than the number of repos we host for now), and double
the others (groups_bysubgroup, permission_sort) since they still had
reasonable cache hit ratios while full. Also alpha-order the
existing cache overrides in our config for improved maintainability.
This will require a Gerrit service restart to take effect, once the
file update has been deployed.
Change-Id: Ieecd1802ce53cc0d37c68476b94b44cbe36fbd6e
The gate testing sets gerrit_database_type to 'h2', but the container
does not setup /var/gerrit/db. It ends up owned by root in the
container and gerrit can't write to it, causing traceback errors for
account_patch_reviews when anything is updated.
Add a db peer directory with the other config dirs, and make sure it
is bind mounted into the image in the same way.
While we are here, alphabetise this list of directories for
consistency.
Change-Id: Ib24e3027622a711db76d24172b27cdebe6152cb6
Currently is is 0700 and owned by root, so the gerrit2 user can't
access the keys in there by default (in production this is already
correct). This is the key used to push replication to gitea for
example.
Change-Id: Ie0230cd74aeb3e759a597c999b2507260f2b6c17
I missed a "stop" command in the rsyslogd capture introduced with
I940a5b05057e832e2efad79d9a2ed5325020ed0c. This means logs were also
falling though and being put into /var/log/syslog as well, which was
not the intent.
Add the stop command so the logs only go to the container logfile.
Change-Id: I8c577fe259de68178404d74e0bea543691af7d87
We're seeing a growing backlog of email events that all must funnel
through the single thread allocated to sending email. We think this may
be related to server slowness that we are observing. Bump the number of
threads to 4 to see if that flushes the queues quicker and gives us a
more responsive server.
Change-Id: I730c8f657191cedb46f81f4abc0e1796ef847b27
This should only land once we are on Gerrit 3.x and happy with it. But
at this point the mysql reviewdb will not be used anymore and config for
it can be removed. We keep general mysql things like tools and backups
in place as the accountPatchReviewDb continues to live in MySQL.
This also comments out calls to jeepyb's welcome-message,
update-blueprint and update-bug entrypoints from the patchset-created
event hook, since they rely on database connections for the moment.
Calls to update-bug in change-abandoned and change-merged event
hooks are retained as those code paths don't rely on database
interaction nor attempt to load the removed configuration.
Change-Id: I6e24dbb223fd3f76954db3dd74a03887cf2e2a8b
Gerrit seems to handle x/ for plugin extensions in polygerrit.
Unfortunately we've got projects called x/* and that breaks cloning of
these projects. Lets just avoid that for nwo until we can do a rename.
Change-Id: Id01739725c22af9d02ac30b1653743b49a35a332
More recent Gerrit versions have replaced teh old drafts feature
with a combination of private changes and work in progress state.
The latter might be useful eventually and could be used to augment
or replace our workflow -1 votes, but the not-so-private nature of
drafts is what caused us to disable them and we should do the same
with private changes as they'll become an attractive nuisance
otherwise.
Change-Id: I213a73b0ba6a3dd2a8ce402d6a396e6c494529c8
This reverts the changes made with
e0fc90cd067647ffcf06e0bfb84fe11636d33be5 as it has been deployed.
Change-Id: If5de429d2259a151c5e4c22fab0c6588341465e1
The hound project has undergone a small re-birth and moved to
https://github.com/hound-search/hound
which has broken our deployment. We've talked about leaving
codesearch up to gitea, but it's not quite there yet. There seems to
be no point working on the puppet now.
This builds a container than runs houndd. It's an opendev specific
container; the config is pulled from project-config directly.
There's some custom scripts that drive things. Some points for
reviewers:
- update-hound-config.sh uses "create-hound-config" (which is in
jeepyb for historical reasons) to generate the config file. It
grabs the latest projects.yaml from project-config and exits with a
return code to indicate if things changed.
- when the container starts, it runs update-hound-config.sh to
populate the initial config. There is a testing environment flag
and small config so it doesn't have to clone the entire opendev for
functional testing.
- it runs under supervisord so we can restart the daemon when
projects are updated. Unlike earlier versions that didn't start
listening till indexing was done, this version now puts up a "Hound
is not ready yet" message when while it is working; so we can drop
all the magic we were doing to probe if hound is listening via
netstat and making Apache redirect to a status page.
- resync-hound.sh is run from an external cron job daily, and does
this update and restart check. Since it only reloads if changes
are made, this should be relatively rare anyway.
- There is a PR to monitor the config file
(https://github.com/hound-search/hound/pull/357) which would mean
the restart is unnecessary. This would be good in the near and we
could remove the cron job.
- playbooks/roles/codesearch is unexciting and deploys the container,
certificates and an apache proxy back to localhost:6080 where hound
is listening.
I've combined removal of the old puppet bits here as the "-codesearch"
namespace was already being used.
Change-Id: I8c773b5ea6b87e8f7dfd8db2556626f7b2500473
As described inline, installing ansible from source now installs the
"ansible-core" package, instead of "ansible-base". Since they can't
live together nicely, we have to do a manual override for the devel
job.
Change-Id: I1299ea330e6de048b661fc087f016491758631c7
When we get to Gerrit 3.x the old html theming, hideci.js and
commentlinks that parse html and rely on urls no longer work. Lets clean
that up when we get there.
We can add back in similar things that polygerrit does support later
but we don't want that to make the upgrade even more difficult to do.
This should only be merged once we are running Gerrit 3.x.
Change-Id: I838840e6cbf09ca28faeb2cf06290e298a4a1f74
These changes are squashed together to simplify applying them to config
management without zuul and ansible running one of these without the
others. We essentially need them all in place at the same time to
accurately reflect the post upgrade state.
We stop blocking /p/ in gerrit's apache vhost. /p/ is used for
dashboards.
We add a few java options that new gerrit sets by default.
We update the gerrit image in docker compose to 3.2.
We update zuul to use basic auth instead of digest auth when talking to
Gerrit.
Change-Id: I6ea38313544ce1ecbc4cfd914b1f33e77d0d2d03
