This includes a switch from the "legacy" style Wildfly-based image
to a new setup using Quarkus.
Because Keycloak maintainers consider H2 databases as a test/dev
only option, there are no good migration and upgrade paths short of
export/import data. Go ahead and change our deployment model to rely
on a proper RDBMS, run locally from a container on the same server.
Change-Id: I01f8045563e9f6db6168b92c5a868b8095c0d97b
This project didn't proceed past the test phase,
let's clean it up.
Revert "Add a functional test for registry.zuul-ci.org"
This reverts commit e701fdd3ca1d798bd912b19e91e154e8a88f43b8.
Revert "Add testinfra for registry.zuul-ci.org"
This reverts commit e00f4e59b39cabc3e33823a957d3623dce06f9c4.
Revert "Add static site for registry.zuul-ci.org"
This reverts commit 31b505d3ba29f751b8f02ff365ee6de6b5d350f9.
Revert "Add SSL cert for registry.zuul-ci.org"
This reverts commit d0a8473d42bb0ee3ab1cc8bffbf5bb2fea90f755.
Change-Id: I1d39306187c7b2d7a908389f88d1a60e1b29ffe3
This uncomments the list additions for the lists.airshipit.org and
lists.katacontainers.io sites on the new mailman server, removing
the configuration for them from the lists.opendev.org server and, in
the case of the latter, removing all our configuration management
for the server as it was the only site hosted there.
Change-Id: Ic1c735469583e922313797f709182f960e691efc
This server has been replaced by insecure-ci-registry02. Remove it. This
will allow us to delete the server as well.
Change-Id: I4d9bc6ab90b13655ace2edf4a6fa7c362623c010
This adds a second registry host. We will remove the other once we've
cut over successfully (should just depend on a DNS update).
Depends-On: https://review.opendev.org/c/opendev/zone-opendev.org/+/886874
Change-Id: Ib6be5ef242ed038c23e0007488f2c21ce10f4fcb
This switches us to running the services against the etherpad group. We
also define vars in a group_vars file rather than a host specific
file. This allows us to switch testing over to etherpad99 to decouple it
from our production hostnames.
A followup change will add a new etherpad production server that will be
deployed alongside the existing one. This refactor makes that a bit
simpler.
Change-Id: I838ad31eb74a3abfd02bbfa77c9c2d007d57a3d4
At this point gitea09-14 should be our only production gitea backends
behind haproxy and the only gitea servers replicated to by Gerrit.
Additionally, our gitea DB backups should be moved to gitea09 by our
depends on change. There shouldn't be any other reason to keep these
servers around as long as the new ones are keeping up.
Depends-On: https://review.opendev.org/c/opendev/system-config/+/876471
Change-Id: I3d4dfa1682ed1d14c6cd9108fbcbe0bf934b72c7
This brings out total of new giteas to 6. We noticed today that load
skyrocketed on the other four new giteas implying that we need more
gitea backends. We think we tracked this down to a bad crawler (doesn't
identify itself as such), but we should be able to handle these
situations more gracefully. Note that gitea14 recycles gitea08's (now
deleted) IP address.
Depends-On: https://review.opendev.org/c/opendev/zone-opendev.org/+/876891
Change-Id: Ia0517eabd507a6e8c92205b894014c05a92380d1
At this point these four servers have been replaced by four new Jammy
servers at gitea09-12. They are no longer behind the opendev.org load
balancer, and Gerrit is not replicating to them. We should remove them
to stop consuming unnecessary resources and avoid any future confusion.
Change-Id: I636b1ed83ed433f3adca2a8b4523335c6a62c702
These servers will replace gitea05-07 and are built on top of Ubuntu
Jammy. Landing this change should deploy a working, but empty, gitea
installation. We will then transplant the brain (db) of gitea01 into
these three new servers so that they know about historical redirects.
Once that is all done we can replicate git content from gerrit to them
and eventually put them behind the production load balancer.
Depends-On: https://review.opendev.org/c/opendev/zone-opendev.org/+/876201
Change-Id: I519564fd16c204ce182bc7cd82d5e638d01a1a6b
This adds a new Jammy Gitea09 server to our inventory. THis should
deploy Gitea and provision empty repos on it for entries in our projects
list. However, we'll need to do database surgery to carry over redirects
from an older server. It is for this reason we don't add the server to
the Gerrit replication list or our load balancer's pool.
We'll take this a step at a time and add the new server to those other
items when it is ready.
Change-Id: Idac0f250f74d8db4ff8d6d68c1a1c35c28a4660f
The mirror in our Limestone Networks donor environment is now
unreachable, but we ceased using this region years ago due to
persistent networking trouble and the admin hasn't been around for
roughly as long, so it's probably time to go ahead and say goodbye
to it.
Change-Id: Ibad440a3e9e5c210c70c14a34bcfec1fb24e07ce
All references to this cloud have been removed from nodepool, so we
can now remove nb03 and the mirror node.
Change-Id: I4d97f7bbb6392656017b1774b413b58bdb797323
This provider is going away and the depends-on change should be the last
step to remove it from nodepool. Once that is complete we can stop
trying to manage the mirror there (it will need to be manually shut
down), stop managing our user accounts, and stop writing cloud.yaml that
include these details for inap/iweb on nodepool nodes.
Note we leave the bridge clouds.yaml content in place so that we can
manually clean up the mirror node. We can safely remove that clouds.yaml
content in the future without much impact.
Depends-On: https://review.opendev.org/c/openstack/project-config/+/867264
Change-Id: I01338712aeae79aa78e7f61d332a2290093c8a1b
This should now be a largely functional deployment of mailman 3. There
are still some bits that need testing but we'll use followup changes to
force failure and hold nodes.
This deployment of mailman3 uses upstream docker container images. We
currently hack up uids and gids to accomodate that. We also hack up the
settings file and bind mount it over the upstream file in order to use
host networking. We override the hyperkitty index type to xapian. All
list domains are hosted in a single installation and we use native
vhosting to handle that.
We'll deploy this to a new server and migrate one mailing list domain at
a time. This will allow us to start with lists.opendev.org and test
things like dmarc settings before expanding to the remaining lists.
A migration script is also included, which has seen extensive
testing on held nodes for importing copies of the production data
sets.
Change-Id: Ic9bf5cfaf0b87c100a6ce003a6645010a7b50358
Similar to Id98768e29a06cebaf645eb75b39e4dc5adb8830d, move the
certificate variables to the group definition file, so that we don't
have to duplicate handlers or definitions for the testing host.
Change-Id: I6650f5621a4969582f40700232a596d84e2b4a06
Currently we define the letsencrypt certs for each host in its
individual host variables.
With recent work we have a trusted CA and SAN names setup in
our testing environment; introducing the possibility that we could
accidentally reference the production host during testing (both have
valid certs, as far as the testing hosts are concerned).
To avoid this, we can use our naming scheme to move our testing hosts
to "99" and avoid collision with the production hosts. As a bonus,
this really makes you think more about your group/host split to get
things right and keep the environment as abstract as possible.
One example of this is that with letsencrypt certificates defined in
host vars, testing and production need to use the same hostname to get
the right certificates created. Really, this should be group-level
information so it applies equally to host01 and host99. To cover
"hostXX.opendev.org" as a SAN we can include the inventory_hostname in
the group variables.
This updates one of the more tricky hosts, static, as a proof of
concept. We rename the handlers to be generic, and update the testing
targets.
Change-Id: Id98768e29a06cebaf645eb75b39e4dc5adb8830d
The etsencrypt_certs variable defined here in the "static" group file
is overwritten by the host variable. This is not doing anything (and
we don't have a logs.openstack.org any more as it is all in object
storage), remove it.
Change-Id: I6910d6652c558c94d71b1609d1194b654bc5b42d
Move the paste testing server to paste99 to distinguish it in testing
from the actual production paste service. Since we have certificates
setup now, we can directly test against "paste99.opendev.org",
removing the insecure flags to various calls.
Change-Id: Ifd5e270604102806736dffa86dff2bf8b23799c5
We've been told these resources are going away. Trying to remove them
gracefully from nodepool. Once that is done we can remove our configs
here.
Depends-On: https://review.opendev.org/c/openstack/project-config/+/831398
Change-Id: I396ca49ab33c09622dd398012528fe7172c39fe8
We're going to want Mailman 3 served over HTTPS for security
reasons, so start by generating certificates for each of the sites
we have in v2. Also collect the acme.sh logs for verification.
Change-Id: I261ae55c6bc0a414beb473abcb30f9a86c63db85
This adds a keycloak server so we can start experimenting with it.
It's based on the docker-compose file Matthieu made for Zuul
(see https://review.opendev.org/819745 )
We should be able to configure a realm and federate with openstackid
and other providers as described in the opendev auth spec. However,
I am unable to test federation with openstackid due its inability to
configure an oauth app at "localhost". Therefore, we will need an
actual deployed system to test it. This should allow us to do so.
It will also allow use to connect realms to the newly available
Zuul admin api on opendev.
It should be possible to configure the realm the way we want, then
export its configuration into a JSON file and then have our playbooks
or the docker-compose file import it. That would allow us to drive
change to the configuration of the system through code review. Because
of the above limitation with openstackid, I think we should regard the
current implementation as experimental. Once we have a realm
configuration that we like (which we will create using the GUI), we
can chose to either continue to maintain the config with the GUI and
appropriate file backups, or switch to a gitops model based on an
export.
My understanding is that all the data (realms configuration and session)
are kept in an H2 database. This is probably sufficient for now and even
production use with Zuul, but we should probably switch to mariadb before
any heavy (eg gerrit, etc) production use.
This is a partial implementation of https://docs.opendev.org/opendev/infra-specs/latest/specs/central-auth.html
We can re-deploy with a new domain when it exists.
Change-Id: I2e069b1b220dbd3e0a5754ac094c2b296c141753
Co-Authored-By: Matthieu Huin <mhuin@redhat.com>
Previously we had set up the test gerrit instance to use the same
hostname as production: review02.opendev.org. This causes some confusion
as we have to override settings specifically for testing like a reduced
heap size, but then also copy settings from the prod host vars as we
override the host vars entirely. Using a new hostname allows us to use a
different set of host vars with unique values reducing confusion.
Change-Id: I4b95bbe1bde29228164a66f2d3b648062423e294
The Open Infrastructure Foundation's developers who maintain the
OpenStackID software are taking over management of the site itself,
and have deployed it on new servers. DNS records have already been
updated to the new IP address, so it's time to clean up our end in
preparation for deleting the old servers we've been running.
OpenStackID is still used by some services we run, like RefStack and
Zanata, and we're still hosting the OpenStackID Git repository and
documentation, so this does not get rid of all references to it.
Change-Id: I1d625d5204f1e9e3a85ba9605465f6ebb9433021
With our system-config-run gerrit/review jobs we have much less need
for a dedicated server to stage changes on. Remove in prepartion of
server cleanup.
Change-Id: I9430f7a2432324a184e3a4f7e41f9e5150c0200c
The paste service needs an upgrade; since others have created a
lodgeit container it seems worth us keeping the service going if only
to maintain the historical corpus of pastes.
This adds the ansible to deploy lodgeit and a sibling mariadb
container. I have imported a dump of the old data as a test. The
dump is ~4gb and imported it takes up about double that; certainly
nothing we need to be too concerned over. The server will be more
than capable of running the db container alongside the lodgeit
instance.
This should have no effect on production until we decide to switch
DNS.
Change-Id: I284864217aa49d664ddc3ebdc800383b2d7e00e3
This installs our Limnoira/meetbot container and configures it on
eavesdrop01.opendev.org. I have ported the configuration from the old
puppet as best I can (it is very verbose); my procedure was to use the
Limnoira wizard to start a new config file then backport everything
from the old file. I felt this was best to not miss any new options.
This does channel logging (via built-in ChannelLogger plugin, along
with a cron job for logs2html) and runs our fork of meetbot.
It exports the channel logs via HTTP to /irclogs and meetings logs to
/meetings. meetings.opendev.org will proxy to these two locations
when the server is active.
Note this has not ported the channel list; so the bot will not be
listening in our channels.
Change-Id: I9f9a466c271e1a706f9f98f816de0e84047519f1
We are trying to replace eavesdrop01.openstack.org
The main landing page serves meeting information which has been moved
to a static site served from AFS at meeting.opendev.org. Redirect
everything to there.
The IRC logs are currently still hosted on eavesdrop01, so while we
work on migrating these, proxy meeting.opendev.org/<irclogs|meetings>
to this server.
Note this will be a no-op until we move the DNS, but we should make
the eavesdrop acme records before merging.
Change-Id: I5c9c23e619dbe930a77f657b5cd6fdd862034301
This site replaces eavesdrop.openstack.org. I think this name makes
more sense.
That is/was being published by jobs directly pushing this onto the
eavesdrop server. Instead, the publishing jobs for irc-meetings now
publish to /afs/openstack.org/project/meetings.opendev.org. This
makes the site available via the static server.
This is actually a production no-op; nothing has changed for the
current publishing. It is still todo to figure out the correct
redirects to keep things working from the existing
eavesdrop.openstack.org and stop the old publishing method.
Depends-On: https://review.opendev.org/c/opendev/zone-opendev.org/+/794085
Change-Id: Ia582c4cee1f074e78cee32626be86fd5eb1d81bd
