1316 Commits

Author SHA1 Message Date
Zuul
89a1edce3d Merge "Remove old debian-ceph mirrors" 2020-10-27 02:57:44 +00:00
Clark Boylan
fa362b813c More old apache acl cleanups
We can rely on Require instead of Order, Allow, Deny, Satisfy since we
are all on apache 2.4 now. This simplifies reasoning about acl rules.

Change-Id: Idedba1558ccaa1c753d1175e356bf26a8d4b1084
2020-10-16 11:16:26 -07:00
Ian Wienand
961bab63d9 Remove old debian-ceph mirrors
The active releases according to [1] are octopus and nautlius.  Remove
the old releases from our mirroring.  This needs manual cleanup of the
jobs and volumes -- I will do this manually as this is mostly about
clearing out old things before moving the mirroring to Ansible.

[1] https://docs.ceph.com/en/latest/releases/

Change-Id: I050f737521fa6837f3b6b52b8028a839a29f7bd2
2020-10-16 14:16:19 +11:00
Mohammed Naser
322afab352 Add ceph octopus mirrors
Change-Id: I8876b89088bf1530c99edd08f644efe03d2cf867
2020-09-10 21:31:16 +00:00
Sorin Sbarnea
ef5645baa7 Improved ask read-only message
Makes the read-only message more detailed and includes hyperlinks.

Change-Id: I01f76949276962971246ce760c371e5bd010cb02
2020-08-24 08:24:07 +01:00
Thierry Carrez
06f725519d Make ask.openstack.org read-only
Nobody maintains our askbot website, and questions there go
unanswered. In the spirit of simplification, make the site
read-only (so that old answers can still be found) and redirect
users to the openstack-discuss mailing-list and Stack Overflow
(which has a decent openstack community answering questions).

Read-only config values documented at:
https://github.com/ASKBOT/askbot-devel/blob/master/askbot/conf/access_control.py

Change-Id: I33d9d7c87a5a17138fcdc37ee8f8b16cda2248d5
2020-08-17 15:07:21 +02:00
Ian Wienand
69a92d0d9d Add arm64 to debian-docker mirroring
Change-Id: Ice04f98131a9a6ab1eb733c93bce713748b84f56
2020-08-14 10:45:55 +10:00
Ian Wienand
45201f3d66 Remove puppet mirror support
Remove the separate "mirror_opendev" group and rename it to just
"mirror".  Update various parts to reflect that change.

We no longer deploy any mirror hosts with puppet, remove the various
configuration files.

Depends-On: https://review.opendev.org/728345
Change-Id: Ia982fe9cb4357447989664f033df976b528aaf84
2020-05-16 10:14:25 +10:00
Ian Wienand
1d0d62c6a6 status.openstack.org: send zuul link to opendev zuul
Due to a configuration issue, zuul.openstack.org is currently throwing
SSL validation errors.  Update the status.openstack.org to the
canonical OpenStack tenant page directly.

Change-Id: Idf08e140de11126061cb6f9783d13dc64fefff60
2020-04-27 09:42:11 +10:00
Zuul
81f9bfd996 Merge "Set AllowEncodedSlashes NoDecode on 8080 revproxy" 2020-04-23 20:12:15 +00:00
Monty Taylor
5468f49254 Remove unused gerrit puppet things
We ain't using em.

Change-Id: I4ce9188a6b6a7e6a670e61bb17ab07e890faebcf
2020-04-19 10:59:25 -05:00
Radosław Piliszek
3c47bdbd5d Set AllowEncodedSlashes NoDecode on 8080 revproxy
This is a follow-up on nocanon from [1].
This ensures Apache does not filter any requests originally
containing encoded slashes.

[1] I5a3a6551536e2d1e87aa074e0de7619a367b1971

Change-Id: I94fcb67a914da6ab4d6e1bfd0a2e02121d22559c
2020-04-05 16:07:28 +02:00
Radosław Piliszek
cc4c46c2e8 Do not canonicalize URLs to registry.npmjs.org
Kolla Monasca Grafana images are currently not buildable in CI
due to:
404 Not Found - GET http://mirror.bhs1.ovh.openstack.org:8080/registry.npmjs/@types%2fcolor-name

The url-escaped slash gets mangled by Apache on its way to
registry which causes 404.

This patch fixes that.

Change-Id: I5a3a6551536e2d1e87aa074e0de7619a367b1971
2020-04-02 14:29:00 +02:00
Zuul
2a2431611d Merge "Remove /tarballs proxy from mirrors" 2020-03-30 21:59:49 +00:00
Ian Wienand
dcc23be55b Remove static site puppet
Remove the puppet related to the static server

Story: #2006598
Task: #36758

Change-Id: I744da83d811e715ff8049eacf3013cd4831e1c49
2020-03-26 10:36:16 +11:00
Ian Wienand
f55580fbf0 Remove files02.openstack.org and related puppet
All this has moved to static.opendev.org; the server can now be
removed.

Change-Id: I8ca5d7a206e950c28bb8372a85b6a62d6b9ba00c
2020-03-26 10:36:13 +11:00
Ian Wienand
7187b313d8 Remove /tarballs proxy from mirrors
As described in the dependent change, which removes the environment
var to set this up, this is no longer required.

Story: #2006598
Task: #39014

Change-Id: I93455dd1512aeb9111feaf516abfb60695976663
Depends-On: https://review.opendev.org/714543
2020-03-24 09:20:01 +11:00
Andreas Jaeger
2c0b82e5e8 Update infra-manual location
The infra-manual now lives on docs.opendev.org, update links.

New location is: https://docs.opendev.org/opendev/infra-manual/latest

Change-Id: I7716c68cbff4f3a640d7161f59cfc034a7ccca52
2020-03-20 22:03:09 +01:00
Clark Boylan
61caec5b77 Use LE cert on review.open*.org
We previously had two manually issued certs (one each for opendev.org
and openstack.org) but now have a single cert with all the appropriate
names in it automatically issued by LE. Use this new cert before the old
one expires.

Change-Id: I635d2bfd820fe138ee951833dd66f157b2b7c097
2020-02-28 08:10:24 -08:00
Ian Wienand
55da1e3d06 Revert "Generate list of 404s for docs.o.o"
This reverts commit c25e91f49632d8e187f35807f250567446bd5102.

This script parses the Apache logs and writes out a local count of the
404 data to files.openstack.org, and then exports it via
files.openstack.org.

As part of the spec [1] we're trying to remove publishing from local
volumes, in general.

Since this is not widely used, there is only one link to it, it's not
discoverable from the landing page of files.openstack.org (which just
shows the afs directory listing), it has a very long latency making it
not that useful for debugging and grepping the logs there have been no
accesses in the past 2 weeks (as far back as logs go) I propose we
remove it.

If we want to retain this, we should publish the output alongside the
docs AFS volume.  That could certainly be done by distributing the
docs keytab to the host and having it write out in a similar cron job.
Another option could be to setup a keypair for remote login and keep
that as a secret in Zuul, and do the same from a periodic job
(complicated by apache logs being root only, so needs some sudo magic
or similar).  Or, we could figure out an altogether better, privacy
respecting client analytics solution.

[1] https://docs.opendev.org/opendev/infra-specs/latest/specs/retire-static.html

Depends-On: https://review.opendev.org/709036
Change-Id: Iccf24a72cf82592bae8c699f9f857aa54fc74f10
2020-02-24 14:43:11 +11:00
James E. Blair
9fead30442 Use LE certs for zuul-ci.org
This switches the zuul-ci.org/zuulci.org vhost to use newly issued
letsencrypt certs.  It also does the same for git.zuul-ci.org, which
is a different vhost.  Since that vhost is tied into a configuration
which can't accept cert file paths (only content), adjust it to use
the newer "website" manifest pattern which can.

Change-Id: I0cd0407754466327147917390c578da336e61269
2020-01-07 15:25:32 -08:00
Marcin Juszkiewicz
1e67561c6b reprepro: sync arm64 packages from Ubuntu cloud archive
This will allow Kolla to run Ubuntu/arm64 CI jobs.

https://review.opendev.org/701121 fails without it.

Change-Id: Ia697fa4ceb8bfb0ee879e167a3b9d7c4b2e50807
2020-01-06 14:59:22 +01:00
Zuul
5d7ad66895 Merge "Further split Debian and Ubuntu reprepro configs" 2019-10-24 17:35:34 +00:00
Zuul
b167038b3f Merge "Mirror Ceph Nautilus for Debian based" 2019-10-23 23:48:37 +00:00
Tobias Urdin
de519d5617 Mirror Ceph Nautilus for Debian based
Ceph Nautilus is released and the official mirror
is available. This adds the Ceph Nautilus mirror
so we can sync it for Stretch and Bionic.

Based on the same change that was done when Mimic
was released [1]

[1] https://review.opendev.org/#/c/571989/

Change-Id: I9424d1f4df58acde8ea70dc16283d4de89189bae
2019-10-24 09:07:30 +11:00
Jeremy Stanley
4f0342be70 Further split Debian and Ubuntu reprepro configs
Sharing an updates file between the Debian and Ubuntu reprepro runs
causes some warnings, and is generally just unclean. They use
different release naming and repositories, so should just have
separate updates files to track them (they're already separate on
the server, they were just being copied from the same source file in
the module).

While here, remove the label and suite parameters from the Debian
reprepro distribution templates, as they're unnecessary and
potentially confusing (job nodes should never be relying on the
suite names as they change at the next release).

Also allow signatures from subkeys of the listed keys to be
sufficient to verify the debian-security mirror's release files,
like we do for the debian mirror.

Change-Id: Id0ff476864f936bbd7c4637f3dc9e2c219c6e465
2019-10-15 20:21:12 +00:00
Kevin Carter
8b0877cb68
Add proxy for quay
This change adds a proxy config for quay which should assist
us when gating using images provided by the publically
available registry.

Change-Id: I971705e59724e70bd9d42a6920cf4f883556f673
Signed-off-by: Kevin Carter <kecarter@redhat.com>
2019-09-19 16:49:20 -05:00
Zuul
e03a97de8f Merge "files.o.o : publish .log as text/plain" 2019-08-22 00:34:44 +00:00
James E. Blair
96aec261da Add logs.opendev.org vhost
This is a near-copy of the vhost template from puppet-openstackci.

Change-Id: I191e41b501629e2cdd82381d66daa3b850e0be81
2019-07-31 14:32:00 -07:00
Ian Wienand
983761213f files.o.o : publish .log as text/plain
This is a follow on to I67870f6d439af2d2a63a5048ef52cecff3e75275 to do
the same for files.openstack.org (as
http://files.openstack.org/mirror/logs/ is a handy central place to
point people at)

Change-Id: I07c707d45ab3e3c6f87460b3346efd7026467c56
2019-07-22 16:32:50 +10:00
Zuul
0f78ac2dcc Merge "Add proxy for registry.access.redhat" 2019-07-11 20:39:34 +00:00
Kevin Carter
525d21a332
Add proxy for registry.access.redhat
This change adds a proxy config for registry.access.redhat which should
assist us when gating using images provided by the publically available
registry.

Change-Id: Ica7477d63659610de852d305a63f3e78d0dd8c4f
Signed-off-by: Kevin Carter <kecarter@redhat.com>
2019-07-10 07:31:08 -05:00
Zuul
d36db889cc Merge "Revert "Move openSUSE Tumbleweed into a caching mirror instead"" 2019-06-27 01:44:50 +00:00
Jeremy Stanley
428872075e Revert "Move openSUSE Tumbleweed into a caching mirror instead"
This reverts commit b3ce1c52dc7ca455ffd94ea07d8a4fb1b6905fa8.
It removed the AFS mirror at the same time it added the proxy,
but jobs don't know to look for the proxy since it's on a
totally different TCP port.

Change-Id: I87cc03eb3322bd7b093dd6fe798aadb48f319805
2019-06-26 23:29:20 +00:00
Clark Boylan
9ea8edc341 Evaluate files website vhosts in context of website not vhost
To deal with puppet scoping fun we evaluate the template for our
files.o.o website vhosts in the context of the website define and not in
the context of httpd::vhost.

Change-Id: I90bb881eb6ad78cede3a8a2548e1dfcf24e1160b
2019-06-06 15:12:15 -07:00
James E. Blair
3c84c65614 Add tarballs.opendev.org vhost
Change-Id: I6aa85bf92b2d5726d3c86b11b103a87f11953c51
2019-06-05 14:10:19 -07:00
Monty Taylor
43fdea4ac4 Remove bugday from status.o.o
It doesn't seem like this is used anymore. Let's remove it before
we update the rest of this, so that we don't have to, you know,
update abandoned things.

Change-Id: I1c3708021046a428da82eaa843961091915ba4af
2019-05-24 14:56:50 -05:00
Dirk Mueller
b3ce1c52dc Move openSUSE Tumbleweed into a caching mirror instead
Tumbleweed is only rarely used in the openStack CI, so mirroring it
fully is not worth the time/space overhead. a caching proxy
should be good enough. Add it to the directories to clean up
and remove the older entries because they will no longer be
matching.

Change-Id: I987da098cf4a7330cdec8da9ae3cfbff2f330bf8
2019-05-24 16:19:40 +10:00
Monty Taylor
e69c7b7fb9 Rename review.openstack.org to review.opendev.org
There are many references to review.openstack.org, and while the
redirect should work, we can also go ahead and fix them.

Change-Id: I28f398796a6392a3dffea1d25cfe2ae3a36a3589
2019-05-09 14:38:51 +00:00
Ian Wienand
2acfc176b0 Remove graphite.openstack.org
The server has been removed, remove it from inventory.

While we're here, s/graphite.openstack.org/graphite.opendev.org/'
... it's a CNAME redirect but we might as well clean up.

Change-Id: I36c951c85316cd65dde748b1e50ffa2e058c9a88
2019-05-08 05:55:33 +10:00
Monty Taylor
b415375f8b Revert "Add cache-control headers for service-types.openstack.org"
This reverts commit fc98399aa4e3837e74b4587df051e9c538fa1462.

Change-Id: Ie9562c096f865195bc44e55ea6425ad207d44e50
2019-05-03 17:20:21 +00:00
Zuul
65e6f62090 Merge "Add cache-control headers for service-types.openstack.org" 2019-05-03 16:25:45 +00:00
Clark Boylan
16fa372e0c Redirect review.openstack.org to review.opendev.org
This is part of the opendev git hosting transition. We do this on
review.opendev.org/review.openstack.org and not files.openstack.org so
that ssh connections continue to work. This will need to be applied
during the maintenance window.

This also updates the canonical urls and logo.

Change-Id: I5bf4dcd6835e379fcdd2d55393c5a844578074a9
2019-04-17 20:50:20 +00:00
Clark Boylan
90e867b0e3 Remove no longer used logstash config
This created confusion when updating configs to handle journald. Remove
the unused files and update docs to point at the proper config location.

Change-Id: Ifd8d8868b124b72a86cf7b5acb30480e72b903ed
2019-04-05 17:29:24 -07:00
Monty Taylor
fc98399aa4 Add cache-control headers for service-types.openstack.org
In implementing the library to consume the service-types data, it became
clear that the behavior could be much more consistent across
implementations if we set cache-control headers. This allows a combined
ETag and time-based approach, so that the data will only be fetched if
it has a stale etag, but it will only be checked for staleness once a
week. Since the data in question is expected to change only rarely, and
then only in additive ways, this should allow pervassive use of the data
without significant cost to the API consumer.

Change-Id: I6de3c79e22fdea9bf70fd725447ca7141af80b50
2019-04-05 15:53:44 -07:00
Ian Wienand
327cfa7893 Skip backports for buster
This is a follow-on to I39cb9dc0aa52cf5b20545baf4acacc21c5459f2a; as
buster has no backports we need to skip this in the reprepro
configuration.  It's a bit hacky, but we can revert when it is
available.

Change-Id: I60e231f23999d0af9c899a30822c71702befb2bd
2019-03-28 10:52:58 +11:00
James E. Blair
9039bdc0b2 Allow all mod_rewrite directives in git-redirect vhosts
For our git redirect virtualhosts, allow the full set of mod_rewrite
directives.  These are entirely under our static control, so should
be safe.

Change-Id: Ia9c12ccc42ea157ebc4e3060841f1ab2d13008a3
2019-03-26 15:09:36 -07:00
Jeremy Stanley
73955d55b7 Set up legacy git redirect sites
As part of the move to Gitea, we're creating compatibility redirects
from our old Cgit and Git HTTP backend sites to opendev.org. This
introduces Apache vhosts for each of git.airshipit.org,
git.openstack.org, git.starlingx.io and git.zuul-ci.org which all
serve the same docroot on the files.openstack.org server. This
docroot houses a single .htaccess file with the relevant redirect
rules.

Change-Id: I729fe39bcbe0a0cae237e9036ed8fa980f897e68
Co-Authored-By: James E. Blair <jeblair@redhat.com>
Task: #29705
2019-03-08 23:54:51 +00:00
Tony Breeds
1da84f5a42 Add Redirect options to static https vhosts
The current static-https-redirect.vhost config doesn't allow publishing
a .htaccess file in order to setup redirects.  We do use redirects on
sites that share data over both http and https.

This change enables the same options for static https sites.

The motivation is to allow release.o.o to use .htacces to provide
static, human friendly URLs for constraints that persist after branch
deletion in the openstack/requirements Repo.

See: http://lists.openstack.org/pipermail/openstack-discuss/2019-February/002682.html
Note: in that discussion I tested with RewriteRule but Redirect work and
that is what I'm proposing.

Change-Id: I4d3abd46eb15d5e116c832e7393ec1ec4cb6866b
2019-02-22 10:26:25 +11:00
James E. Blair
d38fcda214 Improve comments in docker proxy config
This comment was unclear.  Replace it with some potentially more
useful comments.

Change-Id: Ib9215eb33f01e1ed2b4be394d2d43ab72f37ee50
2019-02-21 11:26:49 -08:00