1200 Commits

Author SHA1 Message Date
Thomas Goirand
aa3c7eecbb Fix Debian Release file headers
This fixes Debian's Release file headers when mirrored by reprepro.
Especially, this patch adds NotAutomatic and ButAutomaticUpgrades
headers needed to correctly setup a backport repository.

Change-Id: I5e0c2f11362cdb59bfd0c36ee148bf5d8aab43db
2018-05-29 23:04:16 +02:00
Zuul
a40867b8bf Merge "Add puppetlabs apt puppet5 pool" 2018-05-08 19:43:04 +00:00
Clark Boylan
c25a3bec74 Fix the gerrit-host used by jeepyb config
This template was previously set to use the local hosts fqdn for the
jeepyb gerrit-host config item. Now that we've switched to indexed hosts
for review.o.o that ends up as review01.openstack.org.

This would work fine for network connectivity to the ssh server and
otherwise we wouldn't notice except that this is the value we write into
the .gitreview files. To make them more accurate switch to
review.openstack.org as the explicit value.

Note I considered making this configurable but we already assume so many
other openstack specific values in this file I don't think its super
necessary.

Change-Id: I065af01b2793639453562d924e58ba7c769dcd38
2018-05-03 14:33:51 -07:00
Tobias Urdin
d7b35e66af Add puppetlabs apt puppet5 pool
This adds the puppet5 pools for puppetlabs apt mirror.
We need these two pools to get latest puppet5 versions
on Debian and Ubuntu:

* http://apt.puppetlabs.com/pool/xenial/puppet5/
* http://apt.puppetlabs.com/pool/stretch/puppet5/

The yum puppetlabs mirror is correct and after this fix
we can properly use the latest version available in CI.

Change-Id: Ic5f7a20bcc8181f1dc55dedfaa60c622defe9c43
2018-05-03 11:43:05 +02:00
Zuul
e67c159381 Merge "Use instance variables in puppet template" 2018-05-01 12:19:55 +00:00
Paul Belanger
c027587c9c
Fix 'releases' is deprecated warning
Change-Id: I1b1ab22328df68eea8d2fadf94e4b13a201c9da4
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-04-27 07:53:44 -04:00
Colleen Murphy
ac9a031e32 Stop managing infracloud things
Infracloud is sadly deceased. The upside is we can delete a lot of code
we don't need anymore. This patch removes infracloud nodes from
site.pp so that the puppet-apply test no longer bothers to validate
them, removes the infracloud modules from modules.env so that we don't
bother to install those modules in puppet-apply and puppet functional
tests, and removes the infracloud-specific data from the public hiera.
Additionally stop the puppetmaster from trying to run the infracloud
ansible playbook and finally remove the chocolate region from nodepool's
clouds.yaml (vanilla was already done).

This patch leaves the run_infracloud.sh script and the
infracloud-specific ansible playbooks as well as the infracloud
manifests in the openstack_project puppet module. It's possible those
tools could come in handy in the future if we ever have another
infracloud, and leaving those tools in place doesn't add confusion about
which hosts are actually active nor does it leave cruft that gets
unnecessarily tested.

Change-Id: Ic760cc55f8e17fa7f39f2dd0433f5560aa8e2d65
2018-04-24 09:13:40 +02:00
Colleen Murphy
cdb05c20e4 Use instance variables in puppet template
Using local ruby variables in puppet ERB templates was deprecated in
puppet 3 and doesn't work any more in puppet 4. Access these values as
ruby instance variables instead.

Change-Id: I12f294d353066ff4af211b9823398a5084a585ee
2018-04-20 21:52:41 +02:00
Paul Belanger
346f4297d2
Remove debian-security from reprepro
Currently this is broken on mirror-update.o.o, the reason seems to be
upstream security.debian.org is a separate pool, but we are trying to
add it into a single pool.

This is causing checksum issues on files we download.

For now, remove debian-security and update jobs to pull directly from
security.debian.org.  A follow up patch will move this into its own
pool for reprepro creating mirror.debian-security.

Change-Id: Id918597aecc5114791811c7f30b6109da6a8d0af
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-04-18 13:04:49 -04:00
Paul Belanger
14ec2bd273
round 2: Fix debian-stretch-security for reprepro
Our previous patch didn't work as expected, this is because the repo
data for security is not in the toplevel directory.  This was the
reason for using suite.

This then means we need 2 stanza's for jessie and stretch, since the
suite field only supports a single entry.

Change-Id: I29944191beff261c79f819e434da35e2848877cb
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-04-17 11:35:48 -04:00
Ian Wienand
7d3086c44f Add arm64 to Debian mirror
We want to start building and running this in infra

Change-Id: Idad405d082a513292dad8475b88e34829f57d955
2018-04-04 06:52:31 +10:00
Paul Belanger
758977bf17
Set floating_ip_source for limestone cloud in nodepool
Because of ipv6 support in limestone-region one, we don't actually
need to ask for FIPs.  Stop doing this so we can bring more servers
online.

Change-Id: I48fe176c4a78683834246f59850276133c26e19f
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-03-26 13:34:39 -04:00
David Moreau Simard
cb48653415
Fix bad path for limestone certificate on nodepool servers
Copy/pasta from puppetmaster.o.o meant that we tried to put and use
the certificate in /etc/openstack/ which is not a thing on nodepool
nodes.

Fix that to be the right location and update the nodepool clouds.yaml.

Change-Id: Ib9c17a58fc2aa9ad5eb9091a6a3fd23ff5825731
2018-03-24 15:33:12 -04:00
David Moreau Simard
ead85b623f
Add limestone credentials to nodepool's cloud.yaml
This makes the credentials available so nodepool launcher and builder
can use them.
The limestone credentials have been added to hiera for the nodepool
group.

Change-Id: Idb56db19110e6b30f6231869ff278b90caf99f4b
2018-03-22 18:16:30 -04:00
David Moreau Simard
77e38475fb
Add limestone-specific network configuration to *clouds.yaml
Limestone provides both ipv4 and ipv6 addresses. Add the necessary
configuration to clouds.yaml to make it work.

Change-Id: I3c717132a42f44aaed6568de6780b1972cbf63d0
2018-03-22 15:07:33 -04:00
Paul Belanger
64d7c4e082
Update vexxhost to use raw images
A request from mnaser to use raw images for the provider. There is
also a proposed change to os-client-config to add this, but will need
to be tagged as a new release.

See: https://review.openstack.org/554351

We can revert once os-client-config has been released to pypi.

Change-Id: Ifc3d5656b7b972d5978b4d806c41bf2cc3443842
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-03-19 16:46:24 -04:00
David Moreau Simard
4267298140
Add Limestone credentials to *clouds.yaml
We're happy to welcome Limestone as a new infrastructure donor and
add the following cloud configurations:
- openstackci-limestone (control plane)
- openstackzuul-limestone (nodepool/testing nodes)

The proper credentials have been committed to hiera.

Change-Id: Iba7813f9c90a4ac84fd37ebb6556c6a98c03a079
2018-03-13 18:43:19 -04:00
Zuul
b8051c6eaf Merge "Add ARM64 Ubuntu mirror" 2018-03-06 03:16:35 +00:00
Zuul
e7c529b4b5 Merge "Fix apt-puppetlabs update line" 2018-03-06 03:05:32 +00:00
Ian Wienand
53ac2cfeae Remove per-image logging from nodepool-builder logging config
Change Ia7415d2fbbb320f8eddc4e46c3a055414df5f997 removed per-image
logging from nodepool; it now keeps a series of named log files.  The
"image" logger is removed, along with all the specific build logs.

I972a66f9c153f52df934d25ab2f77596bb2e5429 removed the
image_log_document_root argument; which wasn't doing anything in
nodepool::builder, but was actually being used to fill-out the
builder-logging.conf file.  Thus we need this to not have a bad path
in the config file.

Change-Id: I6fdc3b622454b069f3ad2cf42da584d8df23a110
2018-02-26 14:12:35 +11:00
Ian Wienand
2869020b49 Add linaro cloud to nodepool clouds.yaml
I have added the keys in hiera

Change-Id: Ief447157bb1e14024b8ae5b06dddb8d12b2f62db
2018-02-23 14:05:43 +11:00
Emmet Hikory
011121b073 Add ARM64 Ubuntu mirror
Change-Id: Icdc38691b76fa9418ebd318710b73526e210ca48
2018-02-21 14:52:30 -05:00
Sam Yaple
d3b2b29726
Support docker registry mirrors without URI
Using a URI has not be allowed for many years [0], however this code was
not enforced until 17.10 docker [1].

Sadly, we need to setup 2 different vhost ports for this, old 1.13.1
clients will not work properly if they are combined with new.  Keep
original logic working on 8081 and move 17.10+ clients to 8082.

[0] https://github.com/docker/docker-ce/blob/17.06/components/engine/registry/config.go#L339
[1] 1cc36398f9

Change-Id: I837f562331603a7c00f9ba8554b6aa2975169eeb
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-02-12 13:18:47 -05:00
Paul Belanger
a21fde83ca
Remove tripleo-test-cloud-rh(1|2) support
We longer have any jobs or need to manage VMs in
tripleo-test-cloud-rh(1|2).  This hardware still eventually be removed
so lets also remove it from our configuration.

Change-Id: I588ae945df15beceaf7a60bf6a65b1615b2074f0
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-01-31 15:27:48 -05:00
Paul Belanger
d05e28dbd5
Remove old clouds from nodepool-builder logging
We no longer have infracloud or tripleo-test-cloud-rh1 online, we can
remove them from nodepool-builder logging.

Change-Id: Ie71b5bd1ee56417846b2a62c40dbd2fd4fa8221e
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-01-31 15:25:28 -05:00
Zuul
7f7ffde8a3 Merge "Remove support for snapshot images in nodepool" 2018-01-31 20:00:56 +00:00
Ian Wienand
fb88e521c9 Remove openstack-rax cloud
This is not a cloud we have; there are no details in hiera, and never
have been.

Change-Id: I3cbe23b2d26e4cd0adc6534bf79b8d2fa106d483
2018-01-25 12:13:08 +11:00
Zuul
57d7783470 Merge "Remove infracloud-chocolate from clouds.yaml" 2018-01-24 06:09:05 +00:00
Ian Wienand
aee346a01a Fix apt-puppetlabs update line
Follow-on from Ibf5bd302b121abd64b3c74a6ec0cf8fecc455e6b

The name in the updates file is actually "puppetlabs-debs-pc1".  After
correcting this manually on mirror-update I got a good mirror run.

Change-Id: Icb06102730135ee33cc1dc29024f600e1d09fb04
2018-01-24 15:16:49 +11:00
Zuul
ce149aa353 Merge "Add Puppetlabs mirror for Ubuntu Xenial" 2018-01-24 01:39:37 +00:00
Paul Belanger
1827200ac5
Remove infracloud-chocolate from clouds.yaml
Currently infracloud controllers are down, as such we no longer are
able to get dynamic inventory from it.  Remove the provider for now
and revert if possible.

Change-Id: I5d2c27de04d7036d49e183cbd3ec3063083da42e
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-01-23 17:10:48 -05:00
Zuul
c5a8e22f1b Merge "Add linaro cloud credentials" 2018-01-23 20:02:54 +00:00
Ian Wienand
82732f5efb Add linaro cloud credentials
Linaro has kindly provided us with resources for aarch64 based
testing.  This adds two new providers; openstackci-linaro for control
plane, and openstackzuul-linaro for testing nodes.

Change-Id: I166d9f669ea88663d4ffe70e25a6e908d11cf35f
2018-01-23 11:28:56 +11:00
Zuul
f3cf6085b7 Merge "Add website hosting infrastructure" 2018-01-22 17:33:33 +00:00
Paul Belanger
4ca2a7a598
Remove support for snapshot images in nodepool
Update our nodepool_log_config.py script and remove support for
snapshots.

Generate latest version of nodepool-builder logging.

Change-Id: Ib87e8c352b56f63ca44668f43fa440645fab9f37
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-01-21 18:20:00 -05:00
James E. Blair
7f014a0c3f Add website hosting infrastructure
And the zuul-ci.org website.  Note changes like this need ssl
certs added to private hiera before merging.  Creating the AFS
volume would also be a good idea.

The SSL certs are in hiera, and the AFS volume is mounted.

Change-Id: I1eccee119bf169fea265c53af4c8c016c13c03ec
Depends-On: Ic92726dc341af5802ad803d239bd547ef5068043
Story: 2001382
Task: 6093
2018-01-19 15:38:11 -08:00
Jeremy Stanley
7a1fb8ad1f Allow governance site content to include redirects
Update the overrides allowed for governance.openstack.org so that
the list of governance directories can perform Redirect or
RedirectMatch with .htaccess files instead of needing to involve
system-config reviewers when they want to move content between URLs
within the site.

Change-Id: Ifb70d119588ce8973d0f86fe555f95853259ba06
2018-01-16 16:23:22 +00:00
Jeremy Stanley
2d57c7cfd9 Add a zuul01.openstack.org
In preparation for replacing the zuulv3.openstack.org host with a
larger instance, set up the necessary support in
Puppet/Hiera/Ansible. While we're here, remove or replace old
references to the since-deleted zuul.openstack.org instance, and
where possible update documentation and configuration to refer to
the new zuul.openstack.org CNAME instead of the zuulv3.openstack.org
FQDN so as to smooth the future transition.

Change-Id: Ie51e133afb238dcfdbeff09747cbd2e53093ef84
2018-01-15 20:32:54 +00:00
Paul Belanger
5c78357e9e
Remove infracloud-vanilla from clouds.yaml
The controller for infracloud-vanilla is no longer working, remove
access to it from our services.

Change-Id: Ic2a44f74493d3f328398b67262ae93253714bf67
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-01-12 14:02:05 -05:00
Emilien Macchi
eee900890d Add Puppetlabs mirror for Ubuntu Xenial
Change-Id: Ibf5bd302b121abd64b3c74a6ec0cf8fecc455e6b
2018-01-03 17:19:41 -08:00
Paul Belanger
7f0e0f17cd
Revert "Remove ovh from clouds.yaml files"
This reverts commit fc889368515687a0957002b2b3886b75e4f74aeb.

Change-Id: I0bd6705335f8915cf1e59ffb33c959d634ceedda
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-01-03 10:29:08 -05:00
Paul Belanger
fc88936851
Remove ovh from clouds.yaml files
We are getting authentication errors from OVH at the moment, for now
remove them from clouds.yaml so dynamic inventory from ansible works
again.

Change-Id: Ief1613b281cd51863119b7ce8b6e8dda78badb12
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-01-03 10:26:34 -05:00
Ian Wienand
cec504851d Remove infra-files-ro and infra-files-rw
These are not valid any more and cause issues when deploying new keys

Change-Id: If10b92e722813a5c0d7d71a58563d08571574c19
2017-12-20 11:01:53 +11:00
James E. Blair
4ccf277850 Add an authoritative hidden master
This runs bind as a hidden master nameserver so we can do all the
keysigning there, and then use nsd (or bind) as public authoritative
slaves.

Change-Id: Ifb2ad109103051fa13c4af1c7be1ca0ae98bb1a1
2017-12-19 08:57:33 -08:00
Zuul
e8e341551a Merge "Move api_timeout into clouds.yaml for nodepool" 2017-12-18 20:49:16 +00:00
Zuul
65afee9095 Merge "Add javascript alias to cacti.o.o for xenial" 2017-12-15 01:30:58 +00:00
Paul Belanger
46750e896a
Add javascript alias to cacti.o.o for xenial
It seems there have been some changes to how javascript libraries are
installed in ubuntu xenial. Add an alias to /usr/share/javascript.

Change-Id: I1ea75cd5c9fddc04515414427f9f322d83f14ecb
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-12-14 19:03:42 -05:00
Ian Wienand
7ace3799f1 Remove npm mirroring components
The npm mirror was removed with
Id539d336814cce2ce18898526e561b8b6977f62f.  This change is "inspired"
by a proposed puppet-nodejs update in
Ia7966fb9578d0d79f3a7f9480e3a956555737dc8.  Rather than fixing it up
for the new version, remove it (also, puppet is failing trying to
access /afs/.openstack.org/mirror/npm).

I believe the npmrc.erb file is actually a vestige of prior release
methods and is also no longer required.

Change-Id: I6fa48e4700779d2c90194f0129c770bf2d6d865f
2017-12-15 10:25:18 +11:00
Paul Belanger
a3a2a84d8a
Remove puppetdb / puppetboard server
Currently puppetdb and puppetboard have been broken for some time (+1
year) and with ubuntu precise becoming EOL it is prime for deleting.

This leaves openstack-infra with a gap in reporting for non-root
users.  As such, as proposal is in the works to maybe use ARA.

Change-Id: Ifc73a2dba3b37ebe790a29c0daa948d6bad0aa33
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-12-11 10:11:11 -05:00
Clark Boylan
749ee5405c Timeout logstash es pushes after 300s
By default the elasticsearch output plugin for logstash has no timeout.
I think this is the cause of logstash daemons hanging when we have
elasticsearch node outages. Goal here is to have the timeout force the
request to be retried again (ideally against a different server entirely
but even if it is simply retried to the same server we should operate
properly again once the server that went down is back up).

Change-Id: I0511f32f8ee3f0c40987cab3e2528a85595785b2
2017-12-08 13:48:43 -08:00