2188 Commits

Author SHA1 Message Date
Clark Boylan
2c5bc279d6 Set Gerrit sshd.batchThreads to 0
Currently `gerrit show-queue -w -q` indicates we are only using 2
threads for service users and have 100 available for interactive users.
Unfortunately we really have three current classes of users: Humans,
Zuul, and everything else. We can't currently separate these into
different pools so instead we'll try using a single large pool and
sharing. To do that we set batchThreads to the special value of 0.

Change-Id: I08681a6b88683355ea5780ac452de903c8c8a7a3
2022-02-09 11:04:54 -08:00
Clark Boylan
a2dce17612 Test Gerrit upgrade from 3.4 to 3.5
This reenables Gerrit upgrade testing but tests the 3.4 to 3.5 upgrade
now. Note this may need some work to get happy once we have 3.5 images
which is why we've split it out into a separate change.

Change-Id: Ibbbd3f98ac2df8d99d4ffda57df59f4a47da3cd3
2022-02-03 08:11:31 -08:00
Zuul
5ffc5d6d01 Merge "Add CentOS SIGs for CentOS Stream 9 to AFS mirrors" 2022-02-03 00:14:28 +00:00
Zuul
83ae9c4663 Merge "Manage 10periodic and 20auto-upgrades together" 2022-02-02 16:27:46 +00:00
Alfredo Moralejo
d6102c5405 Add CentOS SIGs for CentOS Stream 9 to AFS mirrors
In CS9 the layout of the repos has changed and the SIGs repos are in a
separatee directory under centos-stream [1] so we need to add a new
rsync command. These repos includes messaging for rabbitmq-server, nfv
for openvswitch and storage for ceph.

[1] http://mirror.stream.centos.org/

Change-Id: I90890aade7ad5f42e15c4c171ed2c2545f2310c4
2022-02-02 17:13:22 +01:00
Zuul
4fa0243458 Merge "Stop mirroring centos-8" 2022-02-02 01:43:35 +00:00
Zuul
8dafc621d7 Merge "Remove gearman from Zuul" 2022-02-01 23:11:30 +00:00
James E. Blair
14f4a20628 Remove gearman from Zuul
Zuul no longer uses gearman, so we can remove the infrastructure
around it.

Change-Id: I3613d812971add4733d3fe509ee22835e5814ec6
2022-02-01 13:52:47 -08:00
Zuul
a54dac41ef Merge "Fix mixed spaces and hard tabs in Gerrit config" 2022-02-01 17:48:41 +00:00
Zuul
cd939ca0a2 Merge "grafana: update docs and make an import script" 2022-02-01 01:25:24 +00:00
Clark Boylan
4ed4488808 Stop mirroring centos-8
This updates our rsync mirroring script for centos to stop mirroring
centos-8. We also delete the 8/ dir if it exists.

Depends-On: https://review.opendev.org/c/openstack/project-config/+/827184
Change-Id: Ida8605b3b2a3cd9ff358745e12714b27dbd9c59b
2022-01-31 13:44:34 -08:00
Zuul
0520fc3856 Merge "grafana: update to oss latest release" 2022-01-31 04:13:17 +00:00
Zuul
4589c8103d Merge "Use grafyaml container image" 2022-01-31 01:18:55 +00:00
Zuul
3009f09826 Merge "Use versioned get-pip.py URL for Ubuntu Bionic" 2022-01-31 00:01:39 +00:00
Jeremy Stanley
e61f584dbc Override DOCTYPE in wheel cache autoindex
As of pip 22.0, its HTML parser no longer accepts any page which
doesn't start with the string "<!DOCTYPE html>" and, unfortunately,
Apache's mod_autoindex declares a very specific HTML 3.2 doctype
instead, causing pip to break any time our wheel cache is added to
its indices. The main index we generate has been updated with
https://review.opendev.org/826969 but we need this change to address
Apache's dynamically generated file lists for that site.

Configure Apache to supply a custom header file for file indices
within the /wheel/ subtree of our mirror vhosts, and alias it from
outside the docroot in order to reduce clutter of the top-level
directory index. Also instruct mod_autoindex to omit its own
document preamble which would otherwise include the original doctype
declaration. Note that this omits the header title and H1 level
headings from the resulting pages, but as these are only meant for
machine parsing anyway and not humans, it's a compromise to keep the
solution as simple and straightforward as possible.

Change-Id: Id71174954b13b80483256d37f773b781f4956c21
2022-01-30 18:54:12 +00:00
Jeremy Stanley
9a86d81abb Use versioned get-pip.py URL for Ubuntu Bionic
Pip 22.0 doesn't support Python versions prior to 3.7, so the
unversioned get-pip.py script refuses to run under Ubuntu Bionic's
default python3 interpreter. Add a 3.6-specific URL instead to work
around this.

Change-Id: Icab5f4dd45d8f290a2f52db083cdc564e5a08776
2022-01-30 15:37:58 +00:00
James E. Blair
535b7162a1 Move Zuul SQL connection to "database"
The sql connection is no longer supported, we need to use "database"
instead.  The corresponding hostvars change has already been made
on bridge.

Change-Id: Ibcac56568f263bd50b2be43baa26c8c514c5272b
2022-01-27 16:46:32 -08:00
Jeremy Stanley
24b9c9c040 Fix mixed spaces and hard tabs in Gerrit config
The gerrit.config template is a gitconfig-like file, so is expected
to use tab characters for indentation. Half the indented lines used
tabs already, so make the rest consistent.

Change-Id: I6e77f0278a25d688b9517c275614485518923bc9
2022-01-27 15:38:37 -08:00
Zuul
8f13c74167 Merge "Fix quoting in zuul-client docker wrapper" 2022-01-27 21:46:14 +00:00
Zuul
a147870bac Merge "Run zuul project rename steps on a single scheduler" 2022-01-27 21:46:11 +00:00
Zuul
7b6d383e07 Merge "reprepro: mirror Ubuntu UCA Yoga" 2022-01-27 18:15:03 +00:00
Zuul
f788e3dc96 Merge "Manage apt.conf.d/20auto-upgrades" 2022-01-27 17:34:03 +00:00
Clark Boylan
6d3a281a42 Manage 10periodic and 20auto-upgrades together
These two apt.conf.d config files are installed by different packages
but have overlap in the configuration they set. Unfortunately if the
wrong one sets the flag to disable periodic updates it wins based on apt
conf's priority rules.

To ensure that we continue to auto update and handle different packages
supplying different config files we manage the entirety of the periodic
config in both of these files at the same time using a common source
file.

Change-Id: I5e408fd7c343adb1de9ec564fe430a6f31ecc360
2022-01-27 09:17:26 -08:00
Dr. Jens Harbott
73d3ec199a Fix quoting in zuul-client docker wrapper
Make sure that arguments are passed quoted. [0]

[0] https://www.gnu.org/software/bash/manual/html_node/Special-Parameters.html

Change-Id: If5f7e97afab36429bd9e02e3f7efc70bf476a022
2022-01-27 13:31:52 +01:00
Marcin Juszkiewicz
bd2cbbc959 reprepro: mirror Ubuntu UCA Yoga
Change-Id: I1e736721b91dc8e85a44356dbf631a22b88d38c3
2022-01-26 18:17:39 +01:00
Ian Wienand
70c1fb2dff grafana: update docs and make an import script
Update the docs to reflect not having grafyaml in the container.

Also move the import into a separate helper script, which can be
manually run on the host if the container needs to be restarted
out-of-band for some reason.

Change-Id: Ib1f6aea7e16180d9b122552a2aa30ce223426941
2022-01-25 14:18:04 +11:00
Clark Boylan
3135018e6b Run zuul project rename steps on a single scheduler
We were using the zuul-scheduler group but there are two schedulers now
and we don't want the zk backups, secret rename, and secret delete to
run twice on two different schedulers. Address this by fixing the rename
playbook to zuul02.opendev.org.

Change-Id: Ic741e97bd4c930cc27db00c2e037dc724a460ef7
2022-01-24 13:07:14 -08:00
Clark Boylan
b88bc56c55 Manage apt.conf.d/20auto-upgrades
This file has been seen on a few servers with the Unattended-Upgrades
flag set to 0 disabling daily unattended upgrades. Most of our servers
have this set to 1 and are fine, but let's go ahead and manage this file
directly to ensure it is always 1 and auto upgrades are enabled.

Note that previously we had been setting this via apt.conf.d/10periodic
which seems to come from the update-notifier-common package on older
systems and is now no longer used. Since that file's prefix is smaller
than 20auto-upgrades the 20auto-ugprades file installed by
unattended-upgrades overrides this value. A future update would be to
coalesce both 10periodic and 20auto-upgrades together into one config
file.

Change-Id: Ic0bdaaf881780072fda7e60ff89b60b3a07b5804
2022-01-24 11:16:35 -08:00
Clark Boylan
9bfacda1ac Upgrade Gerrit to 3.4
The actually upgrade will be performed manually, but this change will be
used to update the docker-compose.yaml file.

If we land this change prior to the upgrade then note the
manage-projects commands will be updated to use the 3.4 image possibly
while gerrit 3.3 is still running. I don't expect this to be a problem
as manage-projects operates via network protocols.

Change-Id: I5775f4518ec48ac984b70820ebd2e645213e702a
2022-01-24 10:54:54 -08:00
Ian Wienand
9c17f9f7f0 grafana: update to oss latest release
It seems the images have split into enterprise and oss releases.  The
OSS release is the one we want.

Since If0d584f848f213aeea385885e3decfaee6303de5 we don't run anything
in the container.  So we can switch to the upstream default of the
Alpine-based container, which is their reccommendation.

Just use the :latest tag.  Generally the API seems pretty stable.  If
we do break, it's better to figure it out quickly rather than pin to
an old version and then have to make huge jumps.

Depends-On: https://review.opendev.org/c/opendev/grafyaml/+/825990
Change-Id: I312a141baf73d750591957197cb5ba829f503fcb
2022-01-24 12:52:06 +11:00
Chandan Kumar (raukadah)
3fd5bba62b Use facebook mirror for CentOS Stream 8
Current mirror http://mirror.dal10.us.leaseweb.net/centos/8-stream
is hitting issues and is impacting all CS8 jobs.

Switching to facebook mirror fixes the issue.

Related-Bug: #1958510

Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
Change-Id: Ibddbee60c4318fd7bd34f281af95a20bf172c572
2022-01-20 15:32:27 +05:30
James E. Blair
1492f22faa Use grafyaml container image
Instead of building a local grafana image with grafyaml installed,
use the plain upstream grafana image along with the newly created
separate opendev grafyaml image to run the dashboards.

Depends-On: https://review.opendev.org/780119
Change-Id: If0d584f848f213aeea385885e3decfaee6303de5
2022-01-20 09:25:00 +11:00
Clark Boylan
6ebbe44528 Fix meetpad audio mute setting
We set the START_AUDIO_MUTE value in our docker compose .env file, but
we didn't pass that value through to the container via the environment
setting in docker-compose.yaml. Fix this so that the jitsi meet config
templating can write out the expected config with the new value.

Change-Id: I3dbebad3ce67a8787ffd31c0db8d9583fe988e50
2022-01-19 11:00:12 -08:00
Clark Boylan
c6da6dcff3 Start meetpad meetings with muted audio
When you connect to meetpad it says:

  You have started the conversation muted

It does this because we have muted video by default. Unfortunately, this
statement is ambiguous because audio is unmuted by default. Address this
by muting audio by default on join as well. Then when you are told you
are muted you can go unmute audio and video if you wish.

Change-Id: Iba399c92e1f8c6fba5e21ad45a2f4c7e5286429c
2022-01-18 13:54:40 -08:00
Tristan Cacqueray
1368b47456 Update gerritbot-matrix version to include ChangeReady event
… as suggested by @avass

Change-Id: Ic3ec594cde80d3016fdacc9ec0be8adaa1e6f77a
2022-01-18 15:49:47 +00:00
Chandan Kumar (raukadah)
47a01011a7 Use facebook mirror for centos stream 9 rsync
Currently dfw.mirror.rackspace.com is used for
syncing CS9 content on OpenDev mirror. But somehow it is
rejecting rsync connections.

Switching to facebook mirror[1] might fix the issue.
[1]. https://admin.fedoraproject.org/mirrormanager/mirrors/CentOS/9-stream/x86_64

Related-Bug: #1957950

Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
Change-Id: I119e468d6b38e4b3a0f73ab0e839f3bba85fd039
2022-01-16 20:18:40 +05:30
Zuul
0c403be6da Merge "Remove opensuse tumbleweed from our mirrors" 2022-01-11 07:40:52 +00:00
Clark Boylan
5cea4cd63f Remove opensuse tumbleweed from our mirrors
Tumbleweed images were an interesting idea to add forward looking
testing of a very new and up to date linux distro. Unfortunately, the
images don't receive the attention they deserve to remain in our CI
system, and nothing seems to use the images anyway. Clean up our
opensuse mirrors and stop mirroring tumbleweed as one step in this clean
up process.

This depends on the change that removes the CI label/images.

Depends-On: https://review.opendev.org/c/openstack/project-config/+/824071
Change-Id: Ie1488b453463da750e1a08536116e44ec129828e
2022-01-10 14:06:46 -08:00
Jeremy Stanley
efd02a405d Monitor the HTTPS cert for id.openinfra.dev
The OpenStackID project has been rebranded, and the old
openstackid.org deployment is being retained temporarily in order to
ease transition, but id.openinfra.dev is in place now and intended
as its successor.

Monitor its HTTPS cert like we do for its predecessor, so we'll be
aware of any impending expiration, as we host some services which
depend on it.

Change-Id: I7fee4d42db672bffa80fbca953979fad9896880e
2022-01-10 21:21:14 +00:00
Jeremy Stanley
3858a0bc23 Move edge-computing ML to opendev Mailman site
The edge-computing discussion list is not OpenStack-specific. It was
originally included on the lists.openstack.org site when we didn't
yet have a more neutral list hosting location. While we're in the
process of moving other non-OpenStack mailing lists off the
lists.openstack.org site, rehome this one to lists.opendev.org by
setting up address forwarding and Web redirects, and moving the
existing mailman list entry for it in our configuration.

Note that this should be a no-op when it merges, as the list move
will be handled manually while deployment is temporarily disabled
for the server.

Change-Id: If5207f0237bee1571924855b769a22d653964af7
2021-12-21 19:28:22 +00:00
Jeremy Stanley
eadb01c5d8 Move summitsponsors ML to openinfra Mailman site
In keeping with its name change to the Open Infrastructure
Foundation, the summit sponsors mailing list is moving from
lists.openstack.org to lists.openinfra.dev. Set up address
forwarding and Web redirects to reflect this, and move the existing
mailman list entry for it in our configuration.

Note that this should be a no-op when it merges, as the list move
will be handled manually while deployment is temporarily disabled
for the server.

Change-Id: I29e1e94885fd16b0edd7001662f367caec591439
2021-12-21 19:20:02 +00:00
Jeremy Stanley
56e8aaa870 Move marketing ML to openinfra Mailman site
In keeping with its name change to the Open Infrastructure
Foundation, the foundation marketing mailing list is moving from
lists.openstack.org to lists.openinfra.dev. Set up address
forwarding and Web redirects to reflect this, and add a mailman list
entry for it (there's no old one to remove as it wasn't previously
included in our configuration).

Note that this should be a no-op when it merges, as the list move
will be handled manually while deployment is temporarily disabled
for the server.

Change-Id: Ibadc4bfc430656286774e25b4dce6d8e29b5acf7
2021-12-21 19:15:09 +00:00
Jeremy Stanley
71d566bf20 Move goldmembers ML to openinfra Mailman site
In keeping with its name change to the Open Infrastructure
Foundation, the foundation gold member mailing list is moving from
lists.openstack.org to lists.openinfra.dev. Set up address
forwarding and Web redirects to reflect this, and add a mailman list
entry for it (there's no old one to remove as it wasn't previously
included in our configuration).

Note that this should be a no-op when it merges, as the list move
will be handled manually while deployment is temporarily disabled
for the server.

Change-Id: I6cd92e052b26705bd16a4b38b3725248cb5691fd
2021-12-21 18:58:16 +00:00
Jeremy Stanley
5d6cee89f4 Move foundation-board-confidential ML to openinfra
In keeping with its name change to the Open Infrastructure
Foundation, the confidential board mailing list is moving from
lists.openstack.org to lists.openinfra.dev. Set up address
forwarding and Web redirects to reflect this, and add a mailman list
entry for it (there's no old one to remove as it wasn't previously
included in our configuration).

Note that this should be a no-op when it merges, as the list move
will be handled manually while deployment is temporarily disabled
for the server.

Change-Id: I191676bcb7f878afab17ec3c1735219d91b4de4d
2021-12-21 18:54:10 +00:00
Jeremy Stanley
558528c66d Move foundation-board ML to openinfra Mailman site
In keeping with its name change to the Open Infrastructure
Foundation, the foundation board mailing list is moving from
lists.openstack.org to lists.openinfra.dev. Set up address
forwarding and Web redirects to reflect this, and add a mailman list
entry for it (there's no old one to remove as it wasn't previously
included in our configuration).

Note that this should be a no-op when it merges, as the list move
will be handled manually while deployment is temporarily disabled
for the server.

Change-Id: Idcac72c067fab66b6322f08c027e9c451a488ca3
2021-12-21 18:50:43 +00:00
Jeremy Stanley
1a6d341a7d Move community ML to openinfra Mailman site
In keeping with its name change to the Open Infrastructure
Foundation, the foundation community mailing list is moving from
lists.openstack.org to lists.openinfra.dev. Set up address
forwarding and Web redirects to reflect this, and add a mailman list
entry for it (there's no old one to remove as it wasn't previously
included in our configuration).

Note that this should be a no-op when it merges, as the list move
will be handled manually while deployment is temporarily disabled
for the server.

Change-Id: I9fff3b920a7fd0f75a3cc7a704003eeb3aab4d8a
2021-12-21 18:46:34 +00:00
Jeremy Stanley
b5583429b2 Move foundation ML to openinfra Mailman site
In keeping with its name change to the Open Infrastructure
Foundation, the general foundation mailing list is moving from
lists.openstack.org to lists.openinfra.dev. Set up address
forwarding and Web redirects to reflect this, and add a mailman list
entry for it (there's no old one to remove as it wasn't previously
included in our configuration).

Note that this should be a no-op when it merges, as the list move
will be handled manually while deployment is temporarily disabled
for the server.

Change-Id: I367dd2a3d9a1c70c14915efa729d643419375060
2021-12-21 18:15:12 +00:00
Jeremy Stanley
b84bdffc08 Add mailman Web redirects for the moved staff list
While the staff mailing list is hidden and private in production,
that configuration is set after creation, so in our deployment tests
we can absolutely verify that HTTP and HTTPS redirects for listinfo
and archives work anyway. This paves the way for any further
rewrites and associated testing we may need to do for other mailing
lists which move between domains, as well as testing redirects we
may set up as part of the v2 to v3 migration.

Change-Id: I68078554a72e3b59d8192ac4339e8654a8351f52
2021-12-21 03:37:10 +00:00
Jeremy Stanley
81f8cdfb7b Add HTTPS vhosts to mailman servers
Add secondary vhosts for HTTPS to each mailman site, but don't
remove the plain HTTP ones for now. Before switching to Mailman 3
we'll replace the current HTTP vhosts with blanket redirects to
HTTPS.

Add tests to make sure this is working, and also add a command-line
test for the lists.openinfra.dev site now that it's got a first
non-default list of its own. Also collect Apache logs from the test
nodes so we can see for sure what might break.

Change-Id: I4d93d643381f17c9a968595587909f0ba3dd6f92
2021-12-20 20:35:14 +00:00
Jeremy Stanley
17e3977eaf Disable default Apache vhost on mailman servers
The apache2 package installs a "default" vhost automatically.
Disable it, since it interferes with vhost matching on the
multi-site lists.openstack.org server. These vhosts are not enabled
on our production servers, so this makes testing more like
production.

Change-Id: I32a3cea034ac0b198ec1f4610cc096a4502306e6
2021-12-20 20:33:57 +00:00