The dependent change enables the "detect-ref" option of hound, which
looks at the remote origin HEAD and indexes on that. That should
allow indexing of our mixed repos that have a mix of "master" and
"main".
Add cirros to the test, which should exercise this path, and take some
screenshosts because this a js/react app and just a "curl" doesn't
help.
Change-Id: I1850577c63566b594f9730f5b8f0bc10b07ff7e4
Depends-On: https://review.opendev.org/c/opendev/jeepyb/+/830919
Once restarted onto the parent change, our Gerrit deployment will no
longer link to Gitiles representations of changes or the Git tree.
Explicitly deny access to the Gitiles URL base path in the Apache
vhost config, since we can't effectively remove the plugin itself.
This should help prevent search engines from finding its copies of
our projects rather than the ones we want people to use in Gitea.
Change-Id: I3c96221256662443f7a43344afd12194dce82b9d
These two apt.conf.d config files are installed by different packages
but have overlap in the configuration they set. Unfortunately if the
wrong one sets the flag to disable periodic updates it wins based on apt
conf's priority rules.
To ensure that we continue to auto update and handle different packages
supplying different config files we manage the entirety of the periodic
config in both of these files at the same time using a common source
file.
Change-Id: I5e408fd7c343adb1de9ec564fe430a6f31ecc360
This file has been seen on a few servers with the Unattended-Upgrades
flag set to 0 disabling daily unattended upgrades. Most of our servers
have this set to 1 and are fine, but let's go ahead and manage this file
directly to ensure it is always 1 and auto upgrades are enabled.
Note that previously we had been setting this via apt.conf.d/10periodic
which seems to come from the update-notifier-common package on older
systems and is now no longer used. Since that file's prefix is smaller
than 20auto-upgrades the 20auto-ugprades file installed by
unattended-upgrades overrides this value. A future update would be to
coalesce both 10periodic and 20auto-upgrades together into one config
file.
Change-Id: Ic0bdaaf881780072fda7e60ff89b60b3a07b5804
While the staff mailing list is hidden and private in production,
that configuration is set after creation, so in our deployment tests
we can absolutely verify that HTTP and HTTPS redirects for listinfo
and archives work anyway. This paves the way for any further
rewrites and associated testing we may need to do for other mailing
lists which move between domains, as well as testing redirects we
may set up as part of the v2 to v3 migration.
Change-Id: I68078554a72e3b59d8192ac4339e8654a8351f52
Add secondary vhosts for HTTPS to each mailman site, but don't
remove the plain HTTP ones for now. Before switching to Mailman 3
we'll replace the current HTTP vhosts with blanket redirects to
HTTPS.
Add tests to make sure this is working, and also add a command-line
test for the lists.openinfra.dev site now that it's got a first
non-default list of its own. Also collect Apache logs from the test
nodes so we can see for sure what might break.
Change-Id: I4d93d643381f17c9a968595587909f0ba3dd6f92
Once the staff mailing list has been migrated to its new Mailman
site, merge this in order to forward posts destined for its old
address to the new one.
Add a test to make sure domain aliases are bein written as expected.
Change-Id: I5fea8e9ee6460417283c0ed7339d0dd447b2ff63
Our deployment tests don't need to send E-mail messages. More to the
point, they may perform actions which would like to send E-mail
messages. Make sure, at the network level, they'll be prevented from
doing so. Also allow all connections to egress from the loopback
interface, so that services like mailman can connect to the Exim MTA
on localhost.
Add new rolevars for egress rules to support this, and also fix up
some missing related vars in the iptables role's documentation.
Change-Id: If4acd2d3d543933ed1e00156cc83fe3a270612bd
Some extra steps are needed to use keycloak with a reverse proxy.
This adjusts the apache config to send the required headers and
the keycloak server config to use them.
Since the openid configuration json page is constructed entirely
from these headers (and not from static configuration), this is
a good test that the entire system is working.
Change-Id: I662dc85836d640cb732f12f39e9a61607767fcf3
This adds a keycloak server so we can start experimenting with it.
It's based on the docker-compose file Matthieu made for Zuul
(see https://review.opendev.org/819745 )
We should be able to configure a realm and federate with openstackid
and other providers as described in the opendev auth spec. However,
I am unable to test federation with openstackid due its inability to
configure an oauth app at "localhost". Therefore, we will need an
actual deployed system to test it. This should allow us to do so.
It will also allow use to connect realms to the newly available
Zuul admin api on opendev.
It should be possible to configure the realm the way we want, then
export its configuration into a JSON file and then have our playbooks
or the docker-compose file import it. That would allow us to drive
change to the configuration of the system through code review. Because
of the above limitation with openstackid, I think we should regard the
current implementation as experimental. Once we have a realm
configuration that we like (which we will create using the GUI), we
can chose to either continue to maintain the config with the GUI and
appropriate file backups, or switch to a gitops model based on an
export.
My understanding is that all the data (realms configuration and session)
are kept in an H2 database. This is probably sufficient for now and even
production use with Zuul, but we should probably switch to mariadb before
any heavy (eg gerrit, etc) production use.
This is a partial implementation of https://docs.opendev.org/opendev/infra-specs/latest/specs/central-auth.html
We can re-deploy with a new domain when it exists.
Change-Id: I2e069b1b220dbd3e0a5754ac094c2b296c141753
Co-Authored-By: Matthieu Huin <mhuin@redhat.com>
Ansible Galaxy indexes tarballs of Ansible roles and collections at
a central site, which in turn points to a dedicated Amazon S3
subdomain. The tools which consume it support overriding the default
Galaxy URL with any arbitrary one, so should be able to take
advantage of this in CI jobs.
Change-Id: Ib5664e5588f7237a19a2cdb6eec3109452e8a107
This is a re-implementation of
I195ebee548071b0b89bd5bf64b251595271178ca that puts 9-stream in a
separate AFS volume
(Note the automated volume name "mirror.centos-stream" comes just
short of the limit)
Change-Id: I483c2982a6931e7d6fc97ab82f7750b72d2ef265
Previous change If91f79a4648920999de8e6bf6e0c9fec82fde233 replaced
one of the instances of yaml.load() in this file with safe_load() in
order to silence what were then warnings. Now they're errors with
current PyYAML, so go ahead and update the other one.
Change-Id: If9f839f60cd71be8be141423ef2b93884d8aeba7
Previously we had set up the test gerrit instance to use the same
hostname as production: review02.opendev.org. This causes some confusion
as we have to override settings specifically for testing like a reduced
heap size, but then also copy settings from the prod host vars as we
override the host vars entirely. Using a new hostname allows us to use a
different set of host vars with unique values reducing confusion.
Change-Id: I4b95bbe1bde29228164a66f2d3b648062423e294
The dependent change exports the ptgbot website on port 8000 in the
container. Proxy this through apache.
Depends-On: https://review.opendev.org/c/openstack/ptgbot/+/812417
Change-Id: Idf9e9f5ffad981427d24a3476c0c1f244721d917
Dockerfile's COPY directive only copies the contents of a directory when
src is a directory. It does not copy the directory itself. This meant
the copy we were using to copy static files put them in /var/gerrit and
not /var/gerrit/static where we need them to be.
Update the Dockerfile to copy to /var/gerrit/static/ to fix this and add
some resource fetching tests to ensure they are served correctly.
Change-Id: I3bb4c06f3d7a57dcfccbbdb27cb8405586949949
In order to avoid unnecessary browser requests to other sites,
install a copy of the OpenDev logo on the Lodgeit server and serve
it from there rather than pointing at one served from Gitea.
Change-Id: I4c3678a1de8ca4a41cd0c64aab71b2e0e25373af
When generically rejecting connections, we'd prefer to signal to
users clearly that it's the firewall rejecting them. For IPv4 we
previously emitted generic ICMP "no route to host" responses, but
this tends to make it look incorrectly like a routing failure.
Switch to flagging our error responses as "administratively
prohibited" which is more accurate and less confusing. We're also
already using icmp6-adm-prohibited for the v6 rules, so this makes
our v4 ruleset more consistent.
Note that the iptables-extensions(8) manpage indicates "Using
icmp-admin-prohibited with kernels that do not support it will
result in a plain DROP instead of REJECT" but all our kernels should
have support for it these days so this isn't a concern.
Change-Id: Id423f3ec03d0c3c4e40ddef34c38f97167b173f6
This switch testing of lists.openstack.org to Focal and we make a CGI
env var update to accomodate newer mailman.
Specifically newer mailman's CGI scripts filter env vars that it will
pass through. We were setting MAILMAN_SITE_DIR to vhost our mailman
installs with apache2, but that doesn't pass the filter and is removed.
HOST is passed through so we update our scripts, apache vhost configs,
exim, and init scripts to use the HOST env var instead.
Change-Id: I5c8c70c219669e37b7b75a61001a2b7f7bb0bb6c
Instead of having testinfra check that gitead serves robots.txt,
let's have it check that apache serves or proxies it (since that's
what the load balancer will see).
Change-Id: I809fe9c5d5b43e73a216b61d72eea95546b9619c
The pastebinit command-line tool hard-codes an allowed list of
pastebin URLs, one of which is "http://paste.openstack.org" so
redirecting to HTTPS and to other hostnames seems to break it.
It has a specific user-agent, so allow plain HTTP access for this
tool, but redirect others.
Change-Id: Ia7c983986e6e9c08299ded5282a83761448b35bb
We merged change I9459e47ecfd19b27b7adcaee9ce91f80d51c124d which
should have opened this port but did not. Add testing for it.
Remove eavesdrop from webservers group
This was overridding the custom iptables ports that were being set
in the eavesdrop group vars file. There appears to be no other use
for the webservers group.
Change-Id: I7109f1472176ff39482f9bdfc8462e5f525f791c
This tests that we can rename both the project and the org the project
lives in. Should just add a bit more robustness to our testing.
Change-Id: I0914e864c787b1dba175e0fabf6ab2648a554d16
It would be useful to test our rename playbook against gitea and gerrit
when we make changes to these related playbooks, roles, and docker
images. To do this we need to converge our test and production setups
for gerrit a bit more. We create an openstack-project-creator account in
the test gerrit to match prod and we have rename_repos.yaml talk to
localhost for gerrit ssh commands.
With that done we can run the rename_repos.yaml playbook from
test-gitea.yaml and test-gerrit.yaml to help ensure the playbook
functions as expected against these services.
Co-Authored-By: Ian Wienand <iwienand@redhat.com>
Change-Id: I49ffaf86828e87705da303f40ad4a86be030c709
This moves review02 out of the review-staging group and into the main
review group. At this point, review01.openstack.org is inactive so we
can remove all references to openstack.org from the groups. We update
the system-config job to run against a focal production server, and
remove the unneeded rsync setup used to move data.
This additionally enables replication; this should be a no-op when
applied as part of the transition process is to manually apply this,
so that DNS setup can pull zone changes from opendev.org.
It also switches to the mysql connector, as noted inline we found some
issues with mariadb.
Note backups follow in a separate step to avoid doing too much at
once, hence dropping the backup group from the testing list.
Change-Id: I7ee3e3051ea8f3237fd5f6bf1dcc3e5996c16d10
Add robots.txt same as the old server. While we're here and before
the db is setup, put everything under /var/lib/lodgeit/ for general
consistency.
Change-Id: Ib40cba5bb9e1f9f98769c00e2e9fe09e8fb2582f
The paste service needs an upgrade; since others have created a
lodgeit container it seems worth us keeping the service going if only
to maintain the historical corpus of pastes.
This adds the ansible to deploy lodgeit and a sibling mariadb
container. I have imported a dump of the old data as a test. The
dump is ~4gb and imported it takes up about double that; certainly
nothing we need to be too concerned over. The server will be more
than capable of running the db container alongside the lodgeit
instance.
This should have no effect on production until we decide to switch
DNS.
Change-Id: I284864217aa49d664ddc3ebdc800383b2d7e00e3
This installs statusbot on eavesdrop01.opendev.org.
Otherwise it's just config translation and bringing up the daemon.
Change-Id: I246b2723372594e65bcd1ba90215d6831d4c0c72
This moves these services to eavesdrop01.opendev.org, a new
Focal-based server to host IRC services.
We have stopped running puppet on eavesdrop01.openstack.org so there
is nothing left for it to do (note the server is still running
meetbot/ptgbot). Remove the commented out puppet run, and remove the
server from puppet groups. Update the host in the Zuul jobs to the
new node.
Change-Id: I809f9af3e78f566362142790f6c79654ef5b8959
