Similar to Id98768e29a06cebaf645eb75b39e4dc5adb8830d, move the
certificate variables to the group definition file, so that we don't
have to duplicate handlers or definitions for the testing host.
Change-Id: I6650f5621a4969582f40700232a596d84e2b4a06
Currently we define the letsencrypt certs for each host in its
individual host variables.
With recent work we have a trusted CA and SAN names setup in
our testing environment; introducing the possibility that we could
accidentally reference the production host during testing (both have
valid certs, as far as the testing hosts are concerned).
To avoid this, we can use our naming scheme to move our testing hosts
to "99" and avoid collision with the production hosts. As a bonus,
this really makes you think more about your group/host split to get
things right and keep the environment as abstract as possible.
One example of this is that with letsencrypt certificates defined in
host vars, testing and production need to use the same hostname to get
the right certificates created. Really, this should be group-level
information so it applies equally to host01 and host99. To cover
"hostXX.opendev.org" as a SAN we can include the inventory_hostname in
the group variables.
This updates one of the more tricky hosts, static, as a proof of
concept. We rename the handlers to be generic, and update the testing
targets.
Change-Id: Id98768e29a06cebaf645eb75b39e4dc5adb8830d
The etsencrypt_certs variable defined here in the "static" group file
is overwritten by the host variable. This is not doing anything (and
we don't have a logs.openstack.org any more as it is all in object
storage), remove it.
Change-Id: I6910d6652c558c94d71b1609d1194b654bc5b42d
Move the paste testing server to paste99 to distinguish it in testing
from the actual production paste service. Since we have certificates
setup now, we can directly test against "paste99.opendev.org",
removing the insecure flags to various calls.
Change-Id: Ifd5e270604102806736dffa86dff2bf8b23799c5
We've been told these resources are going away. Trying to remove them
gracefully from nodepool. Once that is done we can remove our configs
here.
Depends-On: https://review.opendev.org/c/openstack/project-config/+/831398
Change-Id: I396ca49ab33c09622dd398012528fe7172c39fe8
We're going to want Mailman 3 served over HTTPS for security
reasons, so start by generating certificates for each of the sites
we have in v2. Also collect the acme.sh logs for verification.
Change-Id: I261ae55c6bc0a414beb473abcb30f9a86c63db85
This adds a keycloak server so we can start experimenting with it.
It's based on the docker-compose file Matthieu made for Zuul
(see https://review.opendev.org/819745 )
We should be able to configure a realm and federate with openstackid
and other providers as described in the opendev auth spec. However,
I am unable to test federation with openstackid due its inability to
configure an oauth app at "localhost". Therefore, we will need an
actual deployed system to test it. This should allow us to do so.
It will also allow use to connect realms to the newly available
Zuul admin api on opendev.
It should be possible to configure the realm the way we want, then
export its configuration into a JSON file and then have our playbooks
or the docker-compose file import it. That would allow us to drive
change to the configuration of the system through code review. Because
of the above limitation with openstackid, I think we should regard the
current implementation as experimental. Once we have a realm
configuration that we like (which we will create using the GUI), we
can chose to either continue to maintain the config with the GUI and
appropriate file backups, or switch to a gitops model based on an
export.
My understanding is that all the data (realms configuration and session)
are kept in an H2 database. This is probably sufficient for now and even
production use with Zuul, but we should probably switch to mariadb before
any heavy (eg gerrit, etc) production use.
This is a partial implementation of https://docs.opendev.org/opendev/infra-specs/latest/specs/central-auth.html
We can re-deploy with a new domain when it exists.
Change-Id: I2e069b1b220dbd3e0a5754ac094c2b296c141753
Co-Authored-By: Matthieu Huin <mhuin@redhat.com>
Previously we had set up the test gerrit instance to use the same
hostname as production: review02.opendev.org. This causes some confusion
as we have to override settings specifically for testing like a reduced
heap size, but then also copy settings from the prod host vars as we
override the host vars entirely. Using a new hostname allows us to use a
different set of host vars with unique values reducing confusion.
Change-Id: I4b95bbe1bde29228164a66f2d3b648062423e294
The Open Infrastructure Foundation's developers who maintain the
OpenStackID software are taking over management of the site itself,
and have deployed it on new servers. DNS records have already been
updated to the new IP address, so it's time to clean up our end in
preparation for deleting the old servers we've been running.
OpenStackID is still used by some services we run, like RefStack and
Zanata, and we're still hosting the OpenStackID Git repository and
documentation, so this does not get rid of all references to it.
Change-Id: I1d625d5204f1e9e3a85ba9605465f6ebb9433021
With our system-config-run gerrit/review jobs we have much less need
for a dedicated server to stage changes on. Remove in prepartion of
server cleanup.
Change-Id: I9430f7a2432324a184e3a4f7e41f9e5150c0200c
The paste service needs an upgrade; since others have created a
lodgeit container it seems worth us keeping the service going if only
to maintain the historical corpus of pastes.
This adds the ansible to deploy lodgeit and a sibling mariadb
container. I have imported a dump of the old data as a test. The
dump is ~4gb and imported it takes up about double that; certainly
nothing we need to be too concerned over. The server will be more
than capable of running the db container alongside the lodgeit
instance.
This should have no effect on production until we decide to switch
DNS.
Change-Id: I284864217aa49d664ddc3ebdc800383b2d7e00e3
This installs our Limnoira/meetbot container and configures it on
eavesdrop01.opendev.org. I have ported the configuration from the old
puppet as best I can (it is very verbose); my procedure was to use the
Limnoira wizard to start a new config file then backport everything
from the old file. I felt this was best to not miss any new options.
This does channel logging (via built-in ChannelLogger plugin, along
with a cron job for logs2html) and runs our fork of meetbot.
It exports the channel logs via HTTP to /irclogs and meetings logs to
/meetings. meetings.opendev.org will proxy to these two locations
when the server is active.
Note this has not ported the channel list; so the bot will not be
listening in our channels.
Change-Id: I9f9a466c271e1a706f9f98f816de0e84047519f1
We are trying to replace eavesdrop01.openstack.org
The main landing page serves meeting information which has been moved
to a static site served from AFS at meeting.opendev.org. Redirect
everything to there.
The IRC logs are currently still hosted on eavesdrop01, so while we
work on migrating these, proxy meeting.opendev.org/<irclogs|meetings>
to this server.
Note this will be a no-op until we move the DNS, but we should make
the eavesdrop acme records before merging.
Change-Id: I5c9c23e619dbe930a77f657b5cd6fdd862034301
This site replaces eavesdrop.openstack.org. I think this name makes
more sense.
That is/was being published by jobs directly pushing this onto the
eavesdrop server. Instead, the publishing jobs for irc-meetings now
publish to /afs/openstack.org/project/meetings.opendev.org. This
makes the site available via the static server.
This is actually a production no-op; nothing has changed for the
current publishing. It is still todo to figure out the correct
redirects to keep things working from the existing
eavesdrop.openstack.org and stop the old publishing method.
Depends-On: https://review.opendev.org/c/opendev/zone-opendev.org/+/794085
Change-Id: Ia582c4cee1f074e78cee32626be86fd5eb1d81bd
This will provision LE certs for openstackid.org. If we are happy with
the results then the child change can be merged to to swap apache over
to using the new cert.
Change-Id: Icc9fdd8a39630323916d1f33d9867f93fc6f2b85
We have decided to decommision the ask.openstack.org server as it is
running EOL Xenial, and its manually purchased certiface is about to
expire. Although it has been deprecated for some time, we feel like
it has been around long-enough as a resource that it is best if we
replace it with a place-holder. The links included here are the same
as the currently shown header explaining the site is read-only.
There's nowhere particularly relevant to redirect the site, so we add
a static file here, and some minimal Ansible to put it in the right
place in a generic way in-case we want to do the same for another
service.
Change-Id: I8a31f8fcf9b3064c0ae58e463a6014dc14b518a7
This provisions the cert then when we are happy with the results we can
land the child change to swap the cert over in apache.
Change-Id: Id8e66102cf26a3b9819d4638b7589f44f6400634
This provisions the cert but doesn't switch apache to it. When we are
happy with the new cert we can land the child change which will flip
apache over to the new cert.
Change-Id: I9cffd26a51317ea569b078b89cc30dc34c7e7747
This runs the LE ansible alongside the ethercalc puppetry to get an LE
cert provision for this service. Once we are happy with the new cert we
can land the followup change to switch to the LE cert.
Note we don't add an altname for the host because that will require
extra DNS records in rax DNS.
Change-Id: I04c062eb994f672283aa30ffcc0c4d45fc8c50f6
This cleans up zuul01 as it should no longer be used at this point. We
also make the inventory groups a bit more clear that all zuul servers
are under the opendev.org domain now.
Depends-On: https://review.opendev.org/c/opendev/zone-opendev.org/+/790483
Change-Id: I7885fe60028fbd87688f3ae920a24bce4d1a3acd
This zuul02 instance will replace zuul01. There are a few items to
coordinate when doing an actual switch so we haven't removed zuul01 from
inventory here. In particular we need to update gearman server config
values in the zuul cluster and we need to save queues, shutdown zuul01,
then start zuul02's scheduler and restore queues there.
I believe landing this change is safe as we don't appear to start zuul
on new instances by default. Reviewers should double check this.
Depends-On: https://review.opendev.org/c/opendev/zone-opendev.org/+/791039
Change-Id: I524b456e494124d8293fbe8e1468de40f3800772
We are doing this so that we can cleanup the private network + floating
IP setup that the existing mirror does. Once this new mirror is up and
happy we can cname to it and then clean up the old mirror and its
networking config. We do this in order to save an IP that the current
private network router is consuming.
Depends-On: https://review.opendev.org/c/opendev/zone-opendev.org/+/787628
Change-Id: I50c311087c6c28726e36913c7e081f3b3d0ee049
This handles planet.openstack.org and redirects it to the
opendev.org/openstack/planet-openstack repo, where we will put a
README and the OPML file of the last state as we deprecate this
service.
Change-Id: If141aca5efbdbe60c91ceefaa4e05c98cd0ba5bb
The OpenEdge cloud has been offline for five months, initially
disabled in I4e46c782a63279d9c18ff4ba2944c15b3027114b, so go ahead
and clean up lingering references. If it is restored later, this can
be reverted fairly easily.
Depends-On: https://review.opendev.org/783989
Depends-On: https://review.opendev.org/783990
Change-Id: I544895003344bc8202363993b52f978e1c07d061
review02.opendev.org is a much larger replacement server for review01
provided by Vexxhost. It is up and running, with gerrit2 volume
attached and DNS entries.
This adds it to the staging group with no replication and a local h2
database configured for initial bringup. There's quite a bit to
consider for full migration, but this will let us start experimenting.
Change-Id: I3638a5c0c7028dcc800ada42431b75395cff0c42
With our increased ability to test in the gate, there's not much use
for review-dev any more. Remove references.
Change-Id: I97e9865e0b655cd157acf9ffa7d067b150e6fc72
This adds a dockerfile to build an opendevorg/refstack image as well as
the jobs to build and publish it.
Change-Id: Icade6c713fa9bf6ab508fd4d8d65debada2ddb30
The hound project has undergone a small re-birth and moved to
https://github.com/hound-search/hound
which has broken our deployment. We've talked about leaving
codesearch up to gitea, but it's not quite there yet. There seems to
be no point working on the puppet now.
This builds a container than runs houndd. It's an opendev specific
container; the config is pulled from project-config directly.
There's some custom scripts that drive things. Some points for
reviewers:
- update-hound-config.sh uses "create-hound-config" (which is in
jeepyb for historical reasons) to generate the config file. It
grabs the latest projects.yaml from project-config and exits with a
return code to indicate if things changed.
- when the container starts, it runs update-hound-config.sh to
populate the initial config. There is a testing environment flag
and small config so it doesn't have to clone the entire opendev for
functional testing.
- it runs under supervisord so we can restart the daemon when
projects are updated. Unlike earlier versions that didn't start
listening till indexing was done, this version now puts up a "Hound
is not ready yet" message when while it is working; so we can drop
all the magic we were doing to probe if hound is listening via
netstat and making Apache redirect to a status page.
- resync-hound.sh is run from an external cron job daily, and does
this update and restart check. Since it only reloads if changes
are made, this should be relatively rare anyway.
- There is a PR to monitor the config file
(https://github.com/hound-search/hound/pull/357) which would mean
the restart is unnecessary. This would be good in the near and we
could remove the cron job.
- playbooks/roles/codesearch is unexciting and deploys the container,
certificates and an apache proxy back to localhost:6080 where hound
is listening.
I've combined removal of the old puppet bits here as the "-codesearch"
namespace was already being used.
Change-Id: I8c773b5ea6b87e8f7dfd8db2556626f7b2500473
185797a0e5e46fd0f68f7b423e79f732c8541d68 made graphite01 (old server)
accidentally do the container restart; this should be for graphite02.
Change-Id: I881ffecf9af5ee07cc3ebcf34f0e204a6389d16b
This was a host used to transition to docker run nodepool builders. That
transition has been completed for nb01.opendev.org and nb02.opendev.org
and we don't need the third x86 builder.
Change-Id: I93c7fc9b24476527b451415e7c138cd17f3fdf9f
This server is going to be our new arm64 nodepool-builder running on the
new arm64 docker images for nodepool.
Depends-On: https://review.opendev.org/750037
Change-Id: I3b46ff901eb92c7f09b79c22441c3f80bc6f9d15
I forgot in I5b7106e2263010ff353e8a1de43e73b0c0ec57e1 this is a new
mirror, which needs the LE bits setup.
Change-Id: I3109573b2b03453049a265a829445f88f8a87557
The host is review-test.opendev.org, so hostvars for
review-test.openstack.org are not so much going to do anything.
It's easier if we just ssh as root from review to gerrit2
on review-test.
review-test needs to be in letsencrypt group and have a
handler.
We need to install mysql - it's on the existing review
servers but not in ansible, it's just left over from
puppet.
The db credentials are in /root/.gerrit_db.cnf
Change-Id: I90e3c9d1b398cc16fea9f7056cfb059c7140160e
