56 Commits

Author SHA1 Message Date
Clark Boylan
96298bfcc4 Restart apache on graphite when LE updates certs
This was touching a file previously, but we can safely restart apache if
the certs update as this happens non concurrently with puppet updates.

Do this to ensure the cert is kept up to date.

Change-Id: I28168770258c38d13202fad48be3f61ecdc8ec4d
2020-06-03 09:51:23 -07:00
Ian Wienand
4233b79e31 Add limestone opendev.org server
This is to replace the puppet managed openstack.org server

Change-Id: I0e3586befd922cb56d1a0ec9c9cb650add9b225d
Depends-On: https://review.opendev.org/728314
2020-05-16 10:14:25 +10:00
Ian Wienand
a864212b1b Add vexxhost opendev.org mirrors
These are to replace the puppet-based openstack.org mirrors

Depends-On: https://review.opendev.org/728308
Change-Id: Ibdce99daa514fb445f1f8389e7c052ee151057ea
2020-05-16 10:14:25 +10:00
Zuul
5e6732f6a4 Merge "Add focal testing for mirror nodes" 2020-05-13 23:55:12 +00:00
Ian Wienand
7b8b788ce2 Add focal testing for mirror nodes
Change-Id: I64de9a61c5044b93f6ce7e2d31cf51d78fd4ec16
2020-05-13 05:32:54 +10:00
Jeremy Stanley
31acca81d7 Replace OVH CI mirrors
New opendev.org CI mirrors for OVH regions. The old BHS1 mirror was
in the openstack.org domain, so is added new. There was an old GRA1
mirror in the opendev.org domain, so remote it and increment the
ordinal in its short hostname to avoid a collision in the inventory
cache.

This is being done to switch to un-billed flavors in this provider,
to simplify internal billing for their donation of resources.

Change-Id: I05770856b5704aa438ed6bc54ec42ba9efb5cd2a
2020-05-12 19:10:21 +00:00
Ian Wienand
f204337268 Add nb01/nb02 opendev servers
These are replacements for the nb01/02.openstack.org puppet servers

Change-Id: I376d70ee375289b004fb859751743c6fafa21411
2020-05-07 09:10:26 +10:00
Monty Taylor
b23515c623 Make a new dockerized etherpad.opendev.org
Upstream likes building the settings file into the image, but that's
less exciting, let's bind-mount ours in.

Depends-On: https://review.opendev.org/717491/
Change-Id: Ia1894d884ef2a84e1282345b77fe07bf8898f367
2020-04-07 11:10:57 -05:00
James E. Blair
8b093dacd5 Add meetpad server
Depends-On: https://review.opendev.org/714189
Change-Id: I5863aaa805a18f9085ee01c3205b0f9ad602922d
2020-03-25 07:44:24 -07:00
Ian Wienand
ba5d9547c8 Replace nb01.opendev.org with nb04
We are starting over with the container nodepool host, and also moving
it to "nb04" to avoid any possibility of conflicting with the
short-hostname of nb01.openstack.org.

Story: #2007407
Task: #39064

Depends-On: https://review.opendev.org/713575
Depends-On: https://review.opendev.org/713571

Change-Id: I18ab9834ad4da201774e0abef56f618cd7839d36
2020-03-19 07:41:56 +11:00
Zuul
b1576b6020 Merge "Set up LE certs for docs.airshipit.org static site" 2020-03-15 16:19:49 +00:00
Jeremy Stanley
69d0b85775 Set up LE certs for docs.airshipit.org static site
Start installing Let's Encrypt SSL certs on static01 for a new
docs.airshipit.org it will be hosting.

Change-Id: Ia5089515cd02d78267827840521cbee4f71ceb0b
2020-03-15 14:10:28 +00:00
Ian Wienand
dbe0bf1ee6 Add nb01.opendev.org
This configures an opendev nodepool-builder

Change-Id: Id8603d9d7caaac0a1ab935e1c7c80d32b02ae23e
Depends-On: https://review.opendev.org/693118
2020-03-11 09:16:31 +11:00
Ian Wienand
b1bfee423b nodepool-builder: Add webserver
This adds the webserver that serves the logs and generated images.

Change-Id: I230f5291e0bd928af2e00966d76c3f385b749cb6
2020-03-11 09:16:31 +11:00
Jeremy Stanley
4cbdc2fc4d Set up inventory and cert for Open Edge mirror
This adds the Open Edge (formerly Fortnebula) CI mirror.

Change-Id: I1ccf2a602f8a41e00bc64a9516a326cc07d9b254
Depends-On: https://review.opendev.org/711787
2020-03-07 00:24:20 +00:00
Jeremy Stanley
43ed9fc297 Moving FortNebula to OpenEdge
Sister change for Ia5caff34d3fafaffc459e7572a4eef6bd94422ea and
removing earlier references to the mirror server in preparation for
building and adding the new one.

Change-Id: I7d506be85326835d5e77a0c9c461f2d457b1dfd3
2020-03-06 20:43:56 +00:00
Andreas Jaeger
e47de667d5 Kill qa.o.o
This site was never used nor published, it can be killed according to QA
PTL.

codesearch returns no matches for it in any docs.

Keep the occurence in manifests/static.pp, this will get deleted
as part of https://review.opendev.org/710388.

Change-Id: I3c0d3b567a3eccb959dc903f169197e4581f1e13
2020-02-28 09:30:27 +01:00
Ian Wienand
d961b6d0d4 static: implement legacy redirect sites
This is a slight divergence from the accepted spec, where we were
going to implement these redirects via a new haproxy instance
(I961456d44a56f2334d3c94ef27e408f27409cd65).  We've decided it's
easier to keep them on static.opendev.org

The following sites are configured to redirect to whatever they are
redirecting to now on static.opendev.org:

 * devstack.org
 * www.devstack.org
 * ci.openstack.org
 * cinder.openstack.org
 * glance.openstack.org
 * horizon.openstack.org
 * keystone.openstack.org
 * nova.openstack.org
 * qa.openstack.org
 * summit.openstack.org
 * swift.openstack.org

As a bonus, they all get a https instance too, which they didn't have
before.

testinfra coverage should be total for this change.  I have created
the _acme-challange CNAME records for all the above.

Story: #2006598
Task: #38881

Change-Id: I3f1fc108e7bb1c9500ad4d1a51df13bb4ae00cb9
2020-02-27 16:25:39 +11:00
Ian Wienand
b5266ea20c static: provide git services
This creates the redirect sites

 git.airshipit.org
 git.openstack.org
 git.starlingx.io
 git.zuul-ci.org

The htaccess rules are put into the main configuration file to avoid
having to create a directory and manage another file.  We use a macro
to duplicate the rules and retain the old semantics of the http site
redirecting directly (as opposed to doing a extra 301 to
https://git.openstack.org first).  This required adding "/" to the "^"
matches as it now runs in VirtualHost context; no functional change is
intended over the old sites.

This will require _acme-challenge CNAMEs to acme.opendev.org before
being merged.

testinfra is updated to exercise some redirects matching against the
results of the extant sites.

Change-Id: Iaa9d5dc2af3f5f8abc11c2312e4308b50f5fcd2b
2020-02-26 12:27:13 +11:00
Ian Wienand
3206fd02b8 static: move afs sites from files.openstack.org to static.opendev.org
This creates sites to serve

 developer.openstack.org
 docs.openstack.org
 docs.opendev.org
 docs.starlingx.io

which are all just static directories underneath /afs/openstack.org/.

This is currently done by files02.openstack.org, but will be better
served in the future by consolidating in ansible configuration on
static.opendev.org.

The following dns entries need to be made before merging to ensure the
certificates are provisioned

 _acme-challenge.developer.openstack.org
 _acme-challenge.docs.openstack.org
 _acme-challenge.docs.opendev.org
 _acme-challenge.docs.starlingx.io

Once done, we can merge and then cut-over the main DNS entries as we
like.

Since there are some follow-ons, I have not removed the puppet
configuration from files02.openstack.org.  I think it's best we
migrate everything away from that and remove it in one lot.

Change-Id: I459a36f823a8868e6cc09e2b0d85f2fe05d69002
2020-02-21 17:59:14 +01:00
Ian Wienand
047eae459d static: add releases.openstack.org site
This adds the site to publish from

 /afs/openstack.org/project/releases.openstack.org

Change-Id: Ia91deb9a51441ac9974137ed39fc5a185689a11c
Task: #37724
Story: #2006598
2020-02-21 14:35:35 +11:00
Ian Wienand
2f1b2f3eae static: Add service-types.openstack.org
Publishing changes done with https://review.opendev.org/#/c/708518/

Change-Id: I13934473aa85fce17a269f81f67c6332d51a9ab1
Story: #2006598
Task: #37723
2020-02-20 11:09:28 +11:00
Ian Wienand
738468b6ad Add specs.openstack.org
Old content is rsynced and publishing to be switched with
https://review.opendev.org/#/c/708500/

Change-Id: I797bb51970d9e7cd3ee5c2635bb5045c618b9d2c
Story: #2006598
Task: # 37721
2020-02-20 10:37:45 +11:00
Zuul
3308e83885 Merge "Get LE certs for review.o.o" 2020-02-12 18:12:50 +00:00
Monty Taylor
083cbf2911 Get LE certs for review.o.o
We have LE dns entries for review.o.o, but we're not actually
requesting the cert. Go ahead and request it - it'll make the
apache config easier to sort out.

Get the openstack.org certs for review-dev while we're at it.

Change-Id: I91d06c97993ba37204bd1fc326ae823e1b9c0c1a
Depends-On: https://review.opendev.org/707267
Depends-On: https://review.opendev.org/707255
2020-02-11 17:01:43 -06:00
Clark Boylan
0463609584 Add airship-citycloud mirror
This adds a mirror to the new airship citycloud region. Add the host to
the inventory and add necessary host vars for LE setup.

Depends-On: https://review.opendev.org/706573
Change-Id: I33cefe914911b4f5ce5e09e0329ba48e039ede64
2020-02-07 08:55:40 -08:00
Ian Wienand
3fd6e16077 Add tarballs.<openstack|opendev>.org to static.opendev.org
Add these hosts to static.opendev.org, serving from AFS.  Note that
tarballs.openstack.org just redirects to static.opendev.org/openstack.

This should have no effect currently, it will only become live when we
switch DNS.

For more details see the thread at:

 http://lists.openstack.org/pipermail/openstack-infra/2020-January/006584.html

Change-Id: Ie56fac17ffaa91ee55be986de636485a58125a02
2020-02-06 08:24:16 +11:00
Monty Taylor
cc619fe589 Add review-dev01.opendev.org
Add a new review-dev server on the opendev domain with LE support
enabled.

Depends-On: https://review.opendev.org/705661
Change-Id: Ie32124cd617e9986602301f230e83bb138524fdf
2020-02-05 09:58:25 -06:00
Ian Wienand
7ce0d0fb32 Add static01.opendev.org
Add this host for serving content from AFS.

The

 _acme-challenge.governance.openstack.org
 _acme-challenge.security.openstack.org

CNAMES should be in place for creating the certificates (added with
Ie1b92f06b71aa6069fe831b26ba1cc272ce4562c).

Also add a cert for the base server (static.opendev.org) since we
added the DNS entries for it.

Change-Id: I55e0ac7487b02f9a816ac486ed01b73f82b391a5
Story: #2006598
Task: #37757
Depends-On: https://review.opendev.org/704469
2020-01-28 11:30:18 +11:00
Ian Wienand
9a75422145 Add linaro-us mirror
Change-Id: I4abf249fd385872f8ba56c3e41f99d160a68efcd
Story: #2007195
Task: #38358
Depends-On: https://review.opendev.org/703746
2020-01-22 20:17:53 +11:00
Ian Wienand
f5b5ee9336 Add roles for a basic static server
Basic implementation of the opendev static server, described in

 https://docs.opendev.org/opendev/infra-specs/latest/specs/retire-static.html

Change-Id: Ie1b92f06b71aa6069fe831b26ba1cc272ce4562c
Story: #2006598
Task:  #37757
2020-01-16 14:10:08 +11:00
Zuul
b5f9fbc65c Merge "Gracefully restart Apache after cert replacements" 2020-01-14 22:30:15 +00:00
Jeremy Stanley
e5ceab1f7d Gracefully restart Apache after cert replacements
Apache doesn't have a reload, but it has something almost as good: a
"graceful" restart. This begins accepting connections while existing
ones wind down, rather than terminating them prematurely. Most
distributions (including the ones we use) map this to the "reload"
action of their SysV initscripts or SystemD service units for
Apache. As a result, we can be nicer to our users by applying the
"reloaded" state to the service module for it in our Ansible role
when Let's Encrypt SSL certs are replaced.

Change-Id: Iac3fad3d0d8216914d94a42f7705e07cef741847
2020-01-14 20:20:43 +00:00
Clark Boylan
3deef00ba9 Manage insecure-ci-registry cert with LE
This adds a new handler to restart the zuul registry to pick up the new
cert. We may want to consider updating zuul registry to accept a reload
of ssl config without restarting the service.

Depends-On: https://review.opendev.org/702050
Change-Id: I23f6bea68285bc7cb0d12224235eaa16f0d07986
2020-01-13 15:20:20 -08:00
Clark Boylan
f30b39c769 Don't issue cert for zuul01.opendev.org
This name/host doesn't actually exist so don't try issuing a cert for
it. Instead only issue a cert for zuul.opendev.org.

Change-Id: I6c8eaa9280c3d6f070b8a1c79d850ee42e0e8d50
2020-01-10 10:13:02 -08:00
Clark Boylan
3981c02322 Provision LE cert for zuul.opendev.org
This provisions the cert but does not use it yet. We will do the
switchover once the cert is confirmed to be in place.

Depends-On: https://review.opendev.org/701819
Change-Id: I04fee48b9a79758527d8f9e8128c0fa915cd133e
2020-01-09 11:36:41 -08:00
James E. Blair
53338653fd Update zuul-ci.org certs
We have a single vhost for zuul-ci.org and zuulci.org, so we should
request a cert with all 4 hostnames.

We also have a separate vhost to handle the git.zuul-ci.org redirect;
add a cert request for that so we can manage it with LE.

Change-Id: Ia2ba3d3ad4f5ab0356ede371d94af3c77a89eda1
2020-01-07 14:35:25 -08:00
James E. Blair
bb0a118441 Use include_tasks in handlers
The import_tasks lines no longer work in Ansible 2.8.  The tarballs
and logs lines were missed earlier due to in-flight changes.

Change-Id: I0fb2b6fc2260175790617d8fcddb52bc9a6f9ef5
2020-01-07 13:50:09 -08:00
James E. Blair
6288a3c016 Get letsencrypt certs for zuul-ci.org
Change-Id: Ieb0c6d02c11a660c063536206e3f9210796007b8
2020-01-06 08:56:47 -08:00
Clark Boylan
4ab6673092 Add necessary ansible vars for inap mirror LE
This was missed when adding the new inap mirror host to our inventory
and groups.

Change-Id: I02d7088ce1722f0a55fe6b17192fd462028aae5c
2019-11-26 13:52:51 -08:00
Clark Boylan
f7a305afbf Manage opendev.org with LE on all giteas
This catches up gitea02-07 with 01 managing ssl certs with LE.

Change-Id: I06228edca2204c5c57ebc5cb60b9d1308a393058
2019-11-18 12:47:08 -08:00
Clark Boylan
5392f8a27c Manage opendev.org cert with LE
This is the first step in managing the opendev.org cert with LE. We
modify gitea01.opendev.org only to request the cert so that if this
breaks the other 7 giteas can continue to serve opendev.org. When we are
happy with the results we can merge the followup change to update the
other 7 giteas.

Depends-On: https://review.opendev.org/694182
Change-Id: I9587b8c2896975aa0148cc3d9b37f325a0be8970
2019-11-18 12:07:10 -08:00
Ian Wienand
9c7136448a Add mirror01.mtl01.opendev.org
This server is a replacement for the .openstack.org version, which no
longer exists.

Depends-On: https://review.opendev.org/690767
Change-Id: I0d2eeb609219ad96db39d1d59b99ae376419df0e
2019-10-24 11:00:05 +11:00
James E. Blair
48cafd19f8 Add LE cert for logs.opendev.org to static
This can be used in an apache vhost later, but should be fine to
merge now.

Depends-On: https://review.opendev.org/673902
Change-Id: Ic2cb7585433351ec1bdabd88915fa1ca07da44e7
2019-07-31 13:00:50 -07:00
Clark Boylan
4b4eb02f32 Replace the fn mirror again
Networking got weird on the previous host so we rebuilt this one going
back to networking we expect to work (FIPs and all that). This updates
the inventory so that we configure the host properly.

Change-Id: I0dcdbc9efdd330d66b57da0b01d23dd3d747f79b
2019-07-30 15:15:01 -07:00
Jeremy Stanley
b45c672de5 Replace fortnebula mirror
The fortnebula mirror is being rebuilt while the environment there
is under some refactoring. The old mirror isn't reachable any longer
so removing it from our inventory while adding its replacement
should be safe.

Also update the letsencrypt playbooks for the new name.

Change-Id: I789248e4216f4cf059ccc5b071c2a784f9c629e9
2019-07-29 13:08:58 +00:00
Clark Boylan
211aac5a9b Add apache restart handler for fortnebula LE setup
THis was missed previously but we need it for LE to properly restart
apache when certs update.

Change-Id: I6cf498ce1ec8cf5b936d2fedbbfe3c9666483e07
2019-07-03 16:04:58 -07:00
Ian Wienand
7810230408 Add OVH GRA1 mirror
This mirror will be manually configured with kafs (see
https://review.opendev.org/623974).  This should be a nice distant
geographic counterpoint to the IAD RAX server.

This will need to be manually configured with a custom kernel for now,
but fixes are making their way upstream and this host will be
converted when available.

Depends-On: https://review.opendev.org/667529
Change-Id: I6a22933029c096c781c93c33e6edf03bf59223c9
2019-06-27 10:07:44 +10:00
Ian Wienand
d33105535a Separate openafs CI mirror
This is an intermediate step to having both kafs and openafs testing
in the gate; this just makes it clear which host is which.

Change-Id: I8cd006227ed47ad5f2c5eec664083477dd7ba397
2019-06-17 15:56:09 +10:00
Ian Wienand
0041f4f673 Add certificates for IAD/ORD opendev.org mirrors
Change-Id: I509517c7601989cff18e29277f3391a813d5ba99
2019-06-12 11:25:50 +10:00