1482 Commits

Author SHA1 Message Date
Zuul
7a7edb11cd Merge "Add (lack of) license information for org logos" 2022-03-21 00:06:48 +00:00
Zuul
3b123e2726 Merge "gitea: set custom avatars for orgs" 2022-03-18 18:28:30 +00:00
Jeremy Stanley
2f543a539d Add (lack of) license information for org logos
The playbooks/roles/gitea-set-org-logos/files directory potentially
contains files which are not covered by any open source copyright
licenses, do add a document clarifying this.

Change-Id: I55cbc9c768d0c3c467a647aafbc82ece7cae989e
2022-03-18 17:46:04 +00:00
Ian Wienand
2d9c8b620f gitea: set custom avatars for orgs
Over a few upgrades, we've managed to break some of the default avatar
logos you see when browsing code on opendev.org.

After investigating ways to fix this up, we established that there
isn't an exposed API for setting these, but we can do a simple query
to point to logo files on disk.  This implements that.

One caveat is that the logos should be PNG files; particiularly we
note that SVG files don't work reliably because they don't get served
with the image/svg+xml mime-type.

Change-Id: Ie6799de2fb27e09f936c488258dc1bd1c638c370
2022-03-18 11:06:09 +11:00
Jeremy Stanley
4863b1200c Disable partial clone feature in Gitea
Gitea 1.16 added partial clone support, but the clone filters pip
tries to apply (--filter=blob:none) don't work well when combined
with older cgit clients and lead to errors like "Server does not
allow request for unadvertised object" or "protocol error: bad pack
header".

Explicitly disable this feature server-side for now, so that clients
will fall back to making full clones.

Change-Id: Ia86394d5176c28567bf67b60578aadde6629c775
Depends-On: https://review.opendev.org/834196
2022-03-17 16:18:21 +00:00
Zuul
e9e63f1d52 Merge "Clean up Gerrit image builds" 2022-03-16 20:19:20 +00:00
wangxiyuan
a6a5988f8a Fix openEuler mirror problem
openEuler yum mirror in Russia is down. This patch change the
rsync url to the official HongKong one.

This patch also fix the openEuler mirror url nit.

Change-Id: Ifb930e34fd7f16f77ba55bc489e5389c641139de
2022-03-16 12:05:06 +08:00
Ian Wienand
7745bf16f3 grafana: proxy websockets
If you watch the web console of your browser in a grafana page, it
constantly tries to hit /api/live/ws which is currently giving an
error.

Following some combination of [1], [2], [3] and some trial-and-error,
this appears to let apache proxy through the requests.

[1] https://github.com/grafana/grafana/issues/36929
[2] https://github.com/grafana/grafana/issues/34537
[3] https://grafana.com/tutorials/run-grafana-behind-a-proxy/

Change-Id: I6c5ba71a1c0feab36b4df56f80271fa52f6354de
2022-03-10 12:49:56 +11:00
Jeremy Stanley
c43289b75a Correct Apache restart for vexxhost-sjc1 mirror
This typo has apparently been causing occasional deploy job failures
for almost two years.

Change-Id: Ic74fa9241a70c120fc496c4e7461e7c899de90d2
2022-03-08 23:49:48 +00:00
Ian Wienand
c9b580cc0d gitea-haproxy: issue liveness check to HEAD /
By default this sends OPTIONS /, which apache rejects with an error.

Change-Id: Ie5d55b2a2d96a78b34d23cc6fbac62900a23fc37
2022-03-08 09:46:59 +11:00
Ian Wienand
50600f49a2 grafana: set custom home dashboard
Set a home dashboard with a little logo, link to the source files and
a plain list of dashboards.

Change-Id: Ifa9373695c1edb7de83b342948d46a816702ee10
2022-03-07 12:45:03 +11:00
Clark Boylan
f24bbf97a7 Do more robust checks against zuul-web with haproxy
Switch the port 80 and 443 endpoints over to doing http checks instead
of tcp checks. This ensures that both apache and the zuul-web backend
are functional before balancing to them.

The fingergw remains a tcp check.

Change-Id: Iabe2d7822c9ef7e4514b9a0eb627f15b93ad48e2
2022-03-04 14:17:51 -08:00
Zuul
4570e3064e Merge "Adds support for running zuul-registry as a non-root user" 2022-03-04 17:16:02 +00:00
Zuul
c5b95b55fa Merge "Block access to Gitiles" 2022-03-03 22:22:09 +00:00
Jack Morgan
ded27cbb5d Adds support for running zuul-registry as a non-root user
Signed-off-by: Jack Morgan <jack@jento.io>
Change-Id: I89594affb04639b49b409a569036d6afac997251
2022-03-03 09:06:51 -08:00
Clark Boylan
b7ccc12a6b Remove airship-citycloud resources
We've been told these resources are going away. Trying to remove them
gracefully from nodepool. Once that is done we can remove our configs
here.

Depends-On: https://review.opendev.org/c/openstack/project-config/+/831398
Change-Id: I396ca49ab33c09622dd398012528fe7172c39fe8
2022-03-01 11:39:53 -08:00
Zuul
1b8fdec20e Merge "Remove Gerrit's JVM GC logs" 2022-02-28 03:28:49 +00:00
Zuul
36ceb62e51 Merge "Restore is:mergeable predicate in Gerrit" 2022-02-25 20:42:35 +00:00
Clark Boylan
7c9d9d7993 Remove Gerrit's JVM GC logs
These were added when we faced significant memory pressure on the old
server. That is no longer a problem and there is an issue with the
specification that breaks file compression due to destination files
already existing. It seems like the log specification is only able to
rotate once then it cannot keep moving files aside because they already
exist as eg jvm_gc.log.0.gz. This results in annoying errors in the
Gerrit error_log.

Note that it doesn't appear sufficient to remove this log specification
we also need to move the existing jvm_gc.log* files aside or delete
them. This was tested on a held zuul node and I stopped gerrit, updated
the docker-compose file, moved the files aside, then started gerrit and
that got rid of the startup errors in error_log. Merely updating
docker-compose resulted in the same errors on startup.

Change-Id: Ied1464c57b2e8331b9bdf7cbc9ad74f92dea2dfd
2022-02-24 14:41:17 -08:00
Jeremy Stanley
77972b0150 Block access to Gitiles
Once restarted onto the parent change, our Gerrit deployment will no
longer link to Gitiles representations of changes or the Git tree.
Explicitly deny access to the Gitiles URL base path in the Apache
vhost config, since we can't effectively remove the plugin itself.
This should help prevent search engines from finding its copies of
our projects rather than the ones we want people to use in Gitea.

Change-Id: I3c96221256662443f7a43344afd12194dce82b9d
2022-02-18 21:39:27 +00:00
Zuul
3b4e1bd4ed Merge "Use Gitea for Gerrit's code browser URLs" 2022-02-18 21:37:05 +00:00
Jeremy Stanley
0022dc2545 Use Gitea for Gerrit's code browser URLs
This is a reimplementation of earlier change
I8efefe365f3b9ebe97c8c2ce322fa8c6f3b70b3a to link out to Gitea
instead of Gerrit's local Gitiles plugin. This should reduce the
complexity of what we're hosting on the Gerrit server, while at the
same time be less confusing for search engines and users. Configure
the Gitiles plugin to no longer take over Gerrit weblinks, and a
followup change will block access to its URL base path entirely.

Change-Id: I7e194fe5c907b39d53fd0663e06cbfd33a3ae410
2022-02-18 18:50:00 +00:00
Dr. Jens Harbott
1a09a4af6c Restore is:mergeable predicate in Gerrit
Gerrit 3.4.0 stopped generating the is:mergeable predicated by default,
but it seems to be rather helpful for some reviewers. The computational
load caused by this is O(N^2) where N depends on the number of changes
open against a branch and their respective size. Since most of the
changes we process are rather small and also we didn't see a significant
reduction in load when we moved to 3.4, this isn't expected to be an
issue in our installation.

[0] https://www.gerritcodereview.com/3.4.html
[1] https://gerrit-documentation.storage.googleapis.com/Documentation/3.4.0/config-gerrit.html#change.mergeabilityComputationBehavior

Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I9fce11b454255818e4a5817affed5b6e9c19f521
2022-02-18 13:04:39 +01:00
Clark Boylan
7513b5b74f Reload haproxy when its config updates
Without this our config changes are not applying to the running service
until something else reloads or restarts the service.

Change-Id: I4df229d1c42f06159a4b320d4b6a07c5239ca111
2022-02-16 15:30:01 -08:00
Zuul
7dfa0f5fa8 Merge "Haproxy http checks for Gitea" 2022-02-16 22:08:26 +00:00
Clark Boylan
df335525ab Haproxy http checks for Gitea
Previously we were only checking that Apache can open TCP connections to
determine if Gitea is up or down on a backend. This is insufficient
because Gitea itself may be down while Apache is up. In this situation
TCP connection to Apache will function, but if we make an HTTP request
we should get back an error.

To check if both Apache and Gitea are working properly we switch to
using http checks instead. Then if Gitea is down Apache can return a 500
and the Gitea backend will be removed from the pool. Similarly if Apache
is non functional the check will fail to connect via TCP.

Note we don't verify ssl certs for simplicity as checking these in
testing is not straightforward. We didn't have verification with the old
tcp checks so this isn't a regression, but does represent something we
could try and improve in the future.

Change-Id: Id47a1f9028c7575e8fbbd10fabfc9730095cb541
2022-02-15 09:59:52 -08:00
Clark Boylan
34fd1a7604 Stop mirroring Fedora 34
We have removed Fedora 34 from our CI system. Fedora has a short
lifetime for each release, and in this particular case Fedora 34 did not
boot reliably in all our clouds. Now that the images have been removed
we can remove the release from our mirrors too.

Change-Id: I07dfca0ef88dc9531e4cb2c67ebbca5e0503594e
2022-02-14 08:55:30 -08:00
James E. Blair
2a9553ef25 Add Zuul load balancer
This adds a load balancer for zuul-web and fingergw.

Change-Id: Id5aa01151f64f3c85e1532ad66999ef9471c5896
2022-02-10 13:24:42 -08:00
Clark Boylan
2c5bc279d6 Set Gerrit sshd.batchThreads to 0
Currently `gerrit show-queue -w -q` indicates we are only using 2
threads for service users and have 100 available for interactive users.
Unfortunately we really have three current classes of users: Humans,
Zuul, and everything else. We can't currently separate these into
different pools so instead we'll try using a single large pool and
sharing. To do that we set batchThreads to the special value of 0.

Change-Id: I08681a6b88683355ea5780ac452de903c8c8a7a3
2022-02-09 11:04:54 -08:00
Zuul
5ffc5d6d01 Merge "Add CentOS SIGs for CentOS Stream 9 to AFS mirrors" 2022-02-03 00:14:28 +00:00
Zuul
83ae9c4663 Merge "Manage 10periodic and 20auto-upgrades together" 2022-02-02 16:27:46 +00:00
Alfredo Moralejo
d6102c5405 Add CentOS SIGs for CentOS Stream 9 to AFS mirrors
In CS9 the layout of the repos has changed and the SIGs repos are in a
separatee directory under centos-stream [1] so we need to add a new
rsync command. These repos includes messaging for rabbitmq-server, nfv
for openvswitch and storage for ceph.

[1] http://mirror.stream.centos.org/

Change-Id: I90890aade7ad5f42e15c4c171ed2c2545f2310c4
2022-02-02 17:13:22 +01:00
Zuul
4fa0243458 Merge "Stop mirroring centos-8" 2022-02-02 01:43:35 +00:00
Zuul
8dafc621d7 Merge "Remove gearman from Zuul" 2022-02-01 23:11:30 +00:00
James E. Blair
14f4a20628 Remove gearman from Zuul
Zuul no longer uses gearman, so we can remove the infrastructure
around it.

Change-Id: I3613d812971add4733d3fe509ee22835e5814ec6
2022-02-01 13:52:47 -08:00
Zuul
a54dac41ef Merge "Fix mixed spaces and hard tabs in Gerrit config" 2022-02-01 17:48:41 +00:00
Zuul
cd939ca0a2 Merge "grafana: update docs and make an import script" 2022-02-01 01:25:24 +00:00
Clark Boylan
4ed4488808 Stop mirroring centos-8
This updates our rsync mirroring script for centos to stop mirroring
centos-8. We also delete the 8/ dir if it exists.

Depends-On: https://review.opendev.org/c/openstack/project-config/+/827184
Change-Id: Ida8605b3b2a3cd9ff358745e12714b27dbd9c59b
2022-01-31 13:44:34 -08:00
Zuul
0520fc3856 Merge "grafana: update to oss latest release" 2022-01-31 04:13:17 +00:00
Zuul
4589c8103d Merge "Use grafyaml container image" 2022-01-31 01:18:55 +00:00
Zuul
3009f09826 Merge "Use versioned get-pip.py URL for Ubuntu Bionic" 2022-01-31 00:01:39 +00:00
Jeremy Stanley
e61f584dbc Override DOCTYPE in wheel cache autoindex
As of pip 22.0, its HTML parser no longer accepts any page which
doesn't start with the string "<!DOCTYPE html>" and, unfortunately,
Apache's mod_autoindex declares a very specific HTML 3.2 doctype
instead, causing pip to break any time our wheel cache is added to
its indices. The main index we generate has been updated with
https://review.opendev.org/826969 but we need this change to address
Apache's dynamically generated file lists for that site.

Configure Apache to supply a custom header file for file indices
within the /wheel/ subtree of our mirror vhosts, and alias it from
outside the docroot in order to reduce clutter of the top-level
directory index. Also instruct mod_autoindex to omit its own
document preamble which would otherwise include the original doctype
declaration. Note that this omits the header title and H1 level
headings from the resulting pages, but as these are only meant for
machine parsing anyway and not humans, it's a compromise to keep the
solution as simple and straightforward as possible.

Change-Id: Id71174954b13b80483256d37f773b781f4956c21
2022-01-30 18:54:12 +00:00
Jeremy Stanley
9a86d81abb Use versioned get-pip.py URL for Ubuntu Bionic
Pip 22.0 doesn't support Python versions prior to 3.7, so the
unversioned get-pip.py script refuses to run under Ubuntu Bionic's
default python3 interpreter. Add a 3.6-specific URL instead to work
around this.

Change-Id: Icab5f4dd45d8f290a2f52db083cdc564e5a08776
2022-01-30 15:37:58 +00:00
James E. Blair
535b7162a1 Move Zuul SQL connection to "database"
The sql connection is no longer supported, we need to use "database"
instead.  The corresponding hostvars change has already been made
on bridge.

Change-Id: Ibcac56568f263bd50b2be43baa26c8c514c5272b
2022-01-27 16:46:32 -08:00
Jeremy Stanley
24b9c9c040 Fix mixed spaces and hard tabs in Gerrit config
The gerrit.config template is a gitconfig-like file, so is expected
to use tab characters for indentation. Half the indented lines used
tabs already, so make the rest consistent.

Change-Id: I6e77f0278a25d688b9517c275614485518923bc9
2022-01-27 15:38:37 -08:00
Zuul
8f13c74167 Merge "Fix quoting in zuul-client docker wrapper" 2022-01-27 21:46:14 +00:00
Zuul
7b6d383e07 Merge "reprepro: mirror Ubuntu UCA Yoga" 2022-01-27 18:15:03 +00:00
Zuul
f788e3dc96 Merge "Manage apt.conf.d/20auto-upgrades" 2022-01-27 17:34:03 +00:00
Clark Boylan
6d3a281a42 Manage 10periodic and 20auto-upgrades together
These two apt.conf.d config files are installed by different packages
but have overlap in the configuration they set. Unfortunately if the
wrong one sets the flag to disable periodic updates it wins based on apt
conf's priority rules.

To ensure that we continue to auto update and handle different packages
supplying different config files we manage the entirety of the periodic
config in both of these files at the same time using a common source
file.

Change-Id: I5e408fd7c343adb1de9ec564fe430a6f31ecc360
2022-01-27 09:17:26 -08:00
Dr. Jens Harbott
73d3ec199a Fix quoting in zuul-client docker wrapper
Make sure that arguments are passed quoted. [0]

[0] https://www.gnu.org/software/bash/manual/html_node/Special-Parameters.html

Change-Id: If5f7e97afab36429bd9e02e3f7efc70bf476a022
2022-01-27 13:31:52 +01:00