1295 Commits

Author SHA1 Message Date
Marcin Juszkiewicz
1e67561c6b reprepro: sync arm64 packages from Ubuntu cloud archive
This will allow Kolla to run Ubuntu/arm64 CI jobs.

https://review.opendev.org/701121 fails without it.

Change-Id: Ia697fa4ceb8bfb0ee879e167a3b9d7c4b2e50807
2020-01-06 14:59:22 +01:00
Zuul
5d7ad66895 Merge "Further split Debian and Ubuntu reprepro configs" 2019-10-24 17:35:34 +00:00
Zuul
b167038b3f Merge "Mirror Ceph Nautilus for Debian based" 2019-10-23 23:48:37 +00:00
Tobias Urdin
de519d5617 Mirror Ceph Nautilus for Debian based
Ceph Nautilus is released and the official mirror
is available. This adds the Ceph Nautilus mirror
so we can sync it for Stretch and Bionic.

Based on the same change that was done when Mimic
was released [1]

[1] https://review.opendev.org/#/c/571989/

Change-Id: I9424d1f4df58acde8ea70dc16283d4de89189bae
2019-10-24 09:07:30 +11:00
Jeremy Stanley
4f0342be70 Further split Debian and Ubuntu reprepro configs
Sharing an updates file between the Debian and Ubuntu reprepro runs
causes some warnings, and is generally just unclean. They use
different release naming and repositories, so should just have
separate updates files to track them (they're already separate on
the server, they were just being copied from the same source file in
the module).

While here, remove the label and suite parameters from the Debian
reprepro distribution templates, as they're unnecessary and
potentially confusing (job nodes should never be relying on the
suite names as they change at the next release).

Also allow signatures from subkeys of the listed keys to be
sufficient to verify the debian-security mirror's release files,
like we do for the debian mirror.

Change-Id: Id0ff476864f936bbd7c4637f3dc9e2c219c6e465
2019-10-15 20:21:12 +00:00
Kevin Carter
8b0877cb68
Add proxy for quay
This change adds a proxy config for quay which should assist
us when gating using images provided by the publically
available registry.

Change-Id: I971705e59724e70bd9d42a6920cf4f883556f673
Signed-off-by: Kevin Carter <kecarter@redhat.com>
2019-09-19 16:49:20 -05:00
Zuul
e03a97de8f Merge "files.o.o : publish .log as text/plain" 2019-08-22 00:34:44 +00:00
James E. Blair
96aec261da Add logs.opendev.org vhost
This is a near-copy of the vhost template from puppet-openstackci.

Change-Id: I191e41b501629e2cdd82381d66daa3b850e0be81
2019-07-31 14:32:00 -07:00
Ian Wienand
983761213f files.o.o : publish .log as text/plain
This is a follow on to I67870f6d439af2d2a63a5048ef52cecff3e75275 to do
the same for files.openstack.org (as
http://files.openstack.org/mirror/logs/ is a handy central place to
point people at)

Change-Id: I07c707d45ab3e3c6f87460b3346efd7026467c56
2019-07-22 16:32:50 +10:00
Zuul
0f78ac2dcc Merge "Add proxy for registry.access.redhat" 2019-07-11 20:39:34 +00:00
Kevin Carter
525d21a332
Add proxy for registry.access.redhat
This change adds a proxy config for registry.access.redhat which should
assist us when gating using images provided by the publically available
registry.

Change-Id: Ica7477d63659610de852d305a63f3e78d0dd8c4f
Signed-off-by: Kevin Carter <kecarter@redhat.com>
2019-07-10 07:31:08 -05:00
Zuul
d36db889cc Merge "Revert "Move openSUSE Tumbleweed into a caching mirror instead"" 2019-06-27 01:44:50 +00:00
Jeremy Stanley
428872075e Revert "Move openSUSE Tumbleweed into a caching mirror instead"
This reverts commit b3ce1c52dc7ca455ffd94ea07d8a4fb1b6905fa8.
It removed the AFS mirror at the same time it added the proxy,
but jobs don't know to look for the proxy since it's on a
totally different TCP port.

Change-Id: I87cc03eb3322bd7b093dd6fe798aadb48f319805
2019-06-26 23:29:20 +00:00
Clark Boylan
9ea8edc341 Evaluate files website vhosts in context of website not vhost
To deal with puppet scoping fun we evaluate the template for our
files.o.o website vhosts in the context of the website define and not in
the context of httpd::vhost.

Change-Id: I90bb881eb6ad78cede3a8a2548e1dfcf24e1160b
2019-06-06 15:12:15 -07:00
James E. Blair
3c84c65614 Add tarballs.opendev.org vhost
Change-Id: I6aa85bf92b2d5726d3c86b11b103a87f11953c51
2019-06-05 14:10:19 -07:00
Monty Taylor
43fdea4ac4 Remove bugday from status.o.o
It doesn't seem like this is used anymore. Let's remove it before
we update the rest of this, so that we don't have to, you know,
update abandoned things.

Change-Id: I1c3708021046a428da82eaa843961091915ba4af
2019-05-24 14:56:50 -05:00
Dirk Mueller
b3ce1c52dc Move openSUSE Tumbleweed into a caching mirror instead
Tumbleweed is only rarely used in the openStack CI, so mirroring it
fully is not worth the time/space overhead. a caching proxy
should be good enough. Add it to the directories to clean up
and remove the older entries because they will no longer be
matching.

Change-Id: I987da098cf4a7330cdec8da9ae3cfbff2f330bf8
2019-05-24 16:19:40 +10:00
Monty Taylor
e69c7b7fb9 Rename review.openstack.org to review.opendev.org
There are many references to review.openstack.org, and while the
redirect should work, we can also go ahead and fix them.

Change-Id: I28f398796a6392a3dffea1d25cfe2ae3a36a3589
2019-05-09 14:38:51 +00:00
Ian Wienand
2acfc176b0 Remove graphite.openstack.org
The server has been removed, remove it from inventory.

While we're here, s/graphite.openstack.org/graphite.opendev.org/'
... it's a CNAME redirect but we might as well clean up.

Change-Id: I36c951c85316cd65dde748b1e50ffa2e058c9a88
2019-05-08 05:55:33 +10:00
Monty Taylor
b415375f8b Revert "Add cache-control headers for service-types.openstack.org"
This reverts commit fc98399aa4e3837e74b4587df051e9c538fa1462.

Change-Id: Ie9562c096f865195bc44e55ea6425ad207d44e50
2019-05-03 17:20:21 +00:00
Zuul
65e6f62090 Merge "Add cache-control headers for service-types.openstack.org" 2019-05-03 16:25:45 +00:00
Clark Boylan
16fa372e0c Redirect review.openstack.org to review.opendev.org
This is part of the opendev git hosting transition. We do this on
review.opendev.org/review.openstack.org and not files.openstack.org so
that ssh connections continue to work. This will need to be applied
during the maintenance window.

This also updates the canonical urls and logo.

Change-Id: I5bf4dcd6835e379fcdd2d55393c5a844578074a9
2019-04-17 20:50:20 +00:00
Clark Boylan
90e867b0e3 Remove no longer used logstash config
This created confusion when updating configs to handle journald. Remove
the unused files and update docs to point at the proper config location.

Change-Id: Ifd8d8868b124b72a86cf7b5acb30480e72b903ed
2019-04-05 17:29:24 -07:00
Monty Taylor
fc98399aa4 Add cache-control headers for service-types.openstack.org
In implementing the library to consume the service-types data, it became
clear that the behavior could be much more consistent across
implementations if we set cache-control headers. This allows a combined
ETag and time-based approach, so that the data will only be fetched if
it has a stale etag, but it will only be checked for staleness once a
week. Since the data in question is expected to change only rarely, and
then only in additive ways, this should allow pervassive use of the data
without significant cost to the API consumer.

Change-Id: I6de3c79e22fdea9bf70fd725447ca7141af80b50
2019-04-05 15:53:44 -07:00
Ian Wienand
327cfa7893 Skip backports for buster
This is a follow-on to I39cb9dc0aa52cf5b20545baf4acacc21c5459f2a; as
buster has no backports we need to skip this in the reprepro
configuration.  It's a bit hacky, but we can revert when it is
available.

Change-Id: I60e231f23999d0af9c899a30822c71702befb2bd
2019-03-28 10:52:58 +11:00
James E. Blair
9039bdc0b2 Allow all mod_rewrite directives in git-redirect vhosts
For our git redirect virtualhosts, allow the full set of mod_rewrite
directives.  These are entirely under our static control, so should
be safe.

Change-Id: Ia9c12ccc42ea157ebc4e3060841f1ab2d13008a3
2019-03-26 15:09:36 -07:00
Jeremy Stanley
73955d55b7 Set up legacy git redirect sites
As part of the move to Gitea, we're creating compatibility redirects
from our old Cgit and Git HTTP backend sites to opendev.org. This
introduces Apache vhosts for each of git.airshipit.org,
git.openstack.org, git.starlingx.io and git.zuul-ci.org which all
serve the same docroot on the files.openstack.org server. This
docroot houses a single .htaccess file with the relevant redirect
rules.

Change-Id: I729fe39bcbe0a0cae237e9036ed8fa980f897e68
Co-Authored-By: James E. Blair <jeblair@redhat.com>
Task: #29705
2019-03-08 23:54:51 +00:00
Tony Breeds
1da84f5a42 Add Redirect options to static https vhosts
The current static-https-redirect.vhost config doesn't allow publishing
a .htaccess file in order to setup redirects.  We do use redirects on
sites that share data over both http and https.

This change enables the same options for static https sites.

The motivation is to allow release.o.o to use .htacces to provide
static, human friendly URLs for constraints that persist after branch
deletion in the openstack/requirements Repo.

See: http://lists.openstack.org/pipermail/openstack-discuss/2019-February/002682.html
Note: in that discussion I tested with RewriteRule but Redirect work and
that is what I'm proposing.

Change-Id: I4d3abd46eb15d5e116c832e7393ec1ec4cb6866b
2019-02-22 10:26:25 +11:00
James E. Blair
d38fcda214 Improve comments in docker proxy config
This comment was unclear.  Replace it with some potentially more
useful comments.

Change-Id: Ib9215eb33f01e1ed2b4be394d2d43ab72f37ee50
2019-02-21 11:26:49 -08:00
James E. Blair
e3170854f9 Fix docker registry proxy
Our proxy was always returning 404 on content blobs because the
CDN part of the proxy was not matching since it appeared after
a greedier match.

Change-Id: Ie88a0e1f554922f6328809b3d96266cc7a20152b
2019-02-21 11:22:08 -08:00
Clark Boylan
a6703e9e57 Prep for pbx upgrade to xenial
This sets our testing to test pbx manifest on xenial and adds a hiera
group called pbx to the servers so we can properly do servernames with
digits and have common config in hiera.

Change-Id: I8c3096d18fe318c6ca206203de0ac984c8934566
2019-02-13 09:44:48 -08:00
Zuul
d12d145754 Merge "Remove legacy zuul status page" 2019-02-11 20:45:41 +00:00
James E. Blair
1450cb3c7a Allow git-test to use mod_rewrite
To create the gitea redirects, we need more sophisticated tools than
those provide by mod_alias.

Change-Id: I9724bb7af43cf1f50e22ed475d2d4e043b4f1b22
2019-02-06 10:17:55 -08:00
James E. Blair
209e5e03e0 Add git-test website
This serves static files out of afs, so we can test redirects from
git.openstack.org to opendev.

Change-Id: I5bdec079ca12074bb761862632d936306c713df4
2019-02-04 13:06:37 -08:00
Monty Taylor
50580e55d5
Remove legacy zuul status page
We don't use this anymore and it's confusing in git greps.

Change-Id: I4a2fe60c090a0f58e5c23bf793aad8bb8e5fe765
2019-01-17 12:52:56 +00:00
Jeremy Stanley
ea66db19f1 Correct auth_openid scope in survey vhost
With the move to Puppet 4, vhost config templates fail to find
relative references to class parameters within a vhost resource
scope. Be explicit about the scope for the auth_openid parameter in
the survey class from within its associated vhost config template so
that it will be correctly found.

Change-Id: I7c769849449b304193e3eb41d129364817d7db1f
2019-01-16 21:49:17 +00:00
Clark Boylan
130c0a47fc Set CacheIgnoreCacheControl on pypi proxy cache
Newer pip sets cache-control: max-age=0 on requests for pypi indexes.
This tells the proxy cache not to serve the index content from the
cache. Unfortauntely this then means we fail to get this data which
could be cached if we can't talk to the backend for some reason.

By default pypi seems to set a 600 second max-age on these indexes which
should be far better than not caching at all. So set the config to have
apache ignore client side max-age and other cache disabling directives.

Change-Id: I2da7006dcd8a8f4212f5e766ef35ba7b98177ed0
2018-12-13 12:30:43 -08:00
Mohammed Naser
ce5874cba3 Add LXC COPR cache
OpenStack Ansible uses this repo to install LXC on CentOS as they
do not ship with the distribution packages.

They're not so reliable so its' good to get them monitored.

Change-Id: I011c7a0455e4853e582ed437204e94b6bcc73b3f
2018-09-12 12:22:55 -06:00
Mohammed Naser
c33ba6c046 Add missing quote to alpine cache
Change-Id: I6eb2b867e87d6279d4d626613c6a6f87a0125386
2018-09-12 11:42:33 -06:00
Clark Boylan
09288c7c37 Manage clouds.yaml files in ansible
This manages the clouds.yaml files in ansible so that we can get them
updated automatically on bridge.openstack.org (which does not puppet).

Co-Authored-By: James E. Blair <jeblair@redhat.com>
Depends-On: https://review.openstack.org/598378
Change-Id: I2071f2593f57024bc985e18eaf1ffbf6f3d38140
2018-09-04 08:49:00 -07:00
Zuul
40a17586cc Merge "Enable cbs.centos.org mirror" 2018-08-29 18:53:05 +00:00
Mohammed Naser
1534c42409 Adding SJC1 for VEXXHOST
Change-Id: Ia4bf3fcd576d415418f756cb07609b817d035534
2018-08-27 11:34:53 -04:00
Emilien Macchi
96712e0cb8 Enable cbs.centos.org mirror
cbs.centos.org contains some repositories that TripleO project is
interested to pull packages from, having a mirror would ensure more
stability in the gate.

Change-Id: I6337b73703c3cecab714a2fa2de3884c96f1216a
2018-08-16 22:52:03 +02:00
Zuul
05b0fe3cb9 Merge "Mirror puppet5 for Ubuntu Bionic" 2018-08-14 21:47:37 +00:00
Ian Wienand
cbff37f82e Cleanup nodepool.openstack.org to be a zk server
This is just a zookeeper server now, so doesn't need all these
passwords, a cloud.yaml file or a launcher-esque deployment.

Change-Id: I3cb20d9c8af150ecdc1fb0a16208a774e3fb530f
2018-07-30 14:12:33 +10:00
David Shrewsbury
10e4c57128 Use nodepool-builder.logging.conf from puppet-nodepool
Generating a openstack-infra specific version of this file was
deprecated with change I6fdc3b622454b069f3ad2cf42da584d8df23a110 when
build logs was moved out of Python logging.

This means (post the depends-on) the file is exactly as kept in
puppet-nodepool.  Remove the system-config template and just use that.

Depends-On: https://review.openstack.org/586231
Change-Id: I2db37868be12c146678f1c8934db80050c788540
2018-07-30 14:07:53 +10:00
Tobias Urdin
d683d602eb Mirror puppet5 for Ubuntu Bionic
Puppet 5 has now been released officialy [1] for
Ubuntu Bionic so we don't have to use the nightly
builds anymore [2].

This adds bionic to be synced from apt.puppetlabs.com
but also changes the structure of the template and the
releases variable to support the Components param for reprepro.

[1] https://apt.puppetlabs.com/pool/bionic/puppet5/p/puppet-agent/
[2] https://github.com/openstack/puppet-openstack-integration/blob/master/functions#L153

Change-Id: I7a303cfcb4408faad5efb703fb127e923b801a81
2018-07-27 14:20:19 +02:00
Paul Belanger
39d667591e
Set image_format to raw for packethost
To avoid the cost of converting nodepool image from qcow2 to raw on
compute nodes, just upload raw images (not qcow2).

Change-Id: I325af734f843fc618a40ca992a0086ffb1c21cfd
Depends-On: https://review.openstack.org/585838
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-07-25 14:08:55 -04:00
Zuul
8a52cb7e26 Merge "Fix template variables" 2018-07-21 13:40:44 +00:00
Monty Taylor
0d2063cf68
Set vexxhost auth_type to password
There is a bug in sdk that is being fixed that causes auth discovery to
work improperly. Setting auth_type to password works around it.

Change-Id: Id8e8fe82827987660c4dc64bd9cf3d65ffd20cf4
2018-07-18 19:28:06 -05:00