Add a groups-dev node to openstack infra, based on drupal
environment. This node checks out the repository:
https://git.openstack.org/openstack-infra/groups
Build a new deployment from scratch, including drush make
distbuild and drush si auto-installation features.
Change-Id: I71eef1c14724ceb13a896ff768923148dedcc2ef
This is the skeleton we'll fill out with any additional
functionality needed by the salt-trigger slave.
Change-Id: I928c2fa4c64e69140b673615a8c18e4eea79e952
This works by installing PyPy from the Ubuntu PPA, and then invoking
`tox -e pypy` (similar to all the other Python buidlers). PyPy shares
nodes with the Python3 builders.
Change-Id: Ibf683c1995b0f5c04c93c56010befb222b386d7a
* manifests/site.pp: Use distinct manifests for git.o.o and git.o.o
backends. This allows for the haproxy server to not serve git content
and purely be a load balancer.
* modules/cgit/manifests/init.pp: Remove haproxy from cgit module.
Remove stale xinetd cleanup. Select git daemon port when selecting
HTTP(S) ports.
* modules/openstack_project/manifests/git.pp: Make git.pp a manifest to
load balance git servers with haproxy.
* modules/openstack_project/manifests/git_backend.pp: New manifest to
manage servers that actually serve git content. They sit behind a load
balancer.
* modules/openstack_project/manifests/review.pp: Stop replicating repos
to git load balancer.
Change-Id: I343a0d1e0a7b93874c2e2299ed974a3304957efb
* install_modules.sh: Add puppetlabs-haproxy forge module.
* modules/cgit/manifests/init.pp: Add haproxy config to load balance
https, https and git protocol git access. Each git server will host git
http on port 8080, https on port 4443 and git protocol on 29418. These
endpoints will then be load balanced by a single haproxy instance
listening on ports 80, 443 and 9418. The use of haproxy and having
services listen on offset ports to accomodate haproxy is toggleable
using the $balance_git and $behind_proxy boolean flags.
Additionally, configure rsyslog for haproxy.
* modules/cgit/files/rsyslog.haproxy.conf: Enable syslog over UDP on
port 514. This is needed by haproxy to perform logging. Send local0
messages to /var/log/haproxy.log.
* modules/cgit/templates/ssl.conf.erb: Make Apache https listen port
configurable. Remove default virtualhost.
* modules/cgit/templates/httpd.conf.erb: Make Apache http listen port
configurable.
* modules/cgit/templates/git.vhost..erb: Make Apache http(s) listen
ports configuruable. Allow http without redirecting to https as a
fallback option to accomodate CentOS clients.
* modules/openstack_project/manifests/git.pp: Pass load balancer
variables through to the cgit manifest.
* manifests/site.pp: Configure git.o.o to run the load balancer haproxy
and balance across the new gitXX.o.o nodes.
Change-Id: Icefc5923cff9a7c6ce62c1923ec2ea87ebc6474a
The devstack cache steps were being run as the ubuntu user; make
sure they are run as jenkins.
Also update the nodepool configuration file to current values and
include the jenkins credential id.
Change-Id: Icbe8bee3b1800e28034cd36d4d328a6793d55b0e
The pubkey being used previous was the server pubkey, not the
gerrit2 user pubkey. The gerrit2 user pubkey is now in hiera
and switched git.openstack.org to use that for the
authorized_keys file.
Change-Id: I6665ef119071b8b68e4cb6bb7f7e8d79ad04fcbc
Logstash.o.o needs to be given the host and port for its elasticsearch
discover node list. Fix the change that used just a list of hosts.
Change-Id: If421007f633eb54a5dc65d5ede57d1e0a6f1c5d9
Proposal.slave.openstack.org is used to push change proposals into
Gerrit. To do this the slave needs to have the Jenkins' private SSH key
installed. Make it so.
Change-Id: Id3106d3167da0569370000e3c1a724a88abbd245
* manifests/site.pp: Put elasticsearch nodes in a list variable that is
reused.
* modules/logstash/templates/elasticsearch.yml.erb: Update cluster
settings to be more appropriate for the larger cluster size, including
changing the minimum number of master eligible nodes that must be seen
before becoming operational in a cluster. Should help prevent split
brained clusters.
* modules/openstack_project/manifests/cacti.pp: Add new node to cacti
node list.
Change-Id: I9a18422aa56a56b9ad030892a312772d9b6b2a79
The tx slave is going to be used for other jobs that propose changes to
Gerrit. Give it a more appropriate name. This lines up with the move
from hosting this under rackspace legacy to rackspace nova.
Change-Id: I7168d62a4a0aae27d2cd2a7dc6c37595604c8251
Now that Zuul and Jenkins use Gearman we don't need to push Jenkins auth
details down into the zuul manifests. Removes these parameters and vars.
Change-Id: I744977f002eb9fb60c3392f0281ab033ffe2bb12
This calculates the full LOG_PATH in the Zuul config and pases it
to Jenkins. The new path is similar to the old but with the
substitution of a short ZUUL_UUID at the end instead of the Jenkins
build number in order to avoid collisions from multiple Jenkins
masters.
Periodic jobs add a node name to their log path to avoid collisions
from multiple masters. Unfortunately, that value is not accessible
to the logstash worker. This can be solved by having Zuul trigger
periodic jobs.
Add the ZUUL_REF to logstash as "build_ref".
Requires https://review.openstack.org/#/c/39130/
Change-Id: I40bad59e3ad8ed6b4706762ed8b833fd15c13b0d
Add jenkins01 and jenkins02, both of which will have unit test and
devstack slaves. Leave jenkins.o.o alone; over time it will be
reduced so that it alone has special jobs and privileged slaves
attached to it.
Note that currently all of the jobs will be defined on all nodes,
including jobs on timers. I think the long-term fix for that is to
have zuul schedule timed jobs.
Change-Id: I10bbd5555e5194b1031700975d5b3ae6b458b8b3
We keep adding more tempest tests to run per patchset. These tests are
the most expensive to index as they come with many megabytes of
uncompressed logs. Add more logstash worker nodes in an attempt to keep
up.
Change-Id: I41d35792d58d1b0456927b0f5b1b7728bf3d145c
* manifests/site.pp: Unfortunately node names like precise3k-1
match the existing node name pattern precise-?\d+.* and end up with
the precise.slave.openstack.org shared puppet cert as a result.
Switch to precisepy3k which won't have this unintended side effect.
Change-Id: I2a530b0e19af769700e85f9060c2cbd8592eef1d
And use that when constructing log paths and URLs. Use a substr
of the change id or commit sha when constructing URLs so the log
directories are deeper.
Make copying the test results a macro (it's used in several places).
Update the gearman log client to take advantage of the new parameter.
Requires https://review.openstack.org/#/c/36304/
Change-Id: I64faa35eddc4105271efa3de4f83b608b77655c2
* manifests/site.pp: Add a node pattern for slave servers which run
Ubuntu Precise Pangolin and default to Python 3.3 for the Puppet PIP
package provider.
* launch/README: Starting with precise3k slaves, individual Puppet
certificates will now be used per-server.
Change-Id: I2cde440720afd8014fc98dfd133d29a428751a9d
* manifests/site.pp: List the two new elasticsearch nodes in the
appropriate lists.
* modules/logstash/manifests/elasticsearch.pp: Do not restart
elasticsearch when config files change. Service restarts are costly and
should be manually performed when necessary. Otherwise puppet should
simply update the config files.
* modules/logstash/templates/elasticsearch.yml.erb: Update elasticsearch
config with new cluster topology. Increase memory available for
indexing.
* modules/openstack_project/manifests/cacti.pp: Add new nodes to cacti
monitoring list.
Adding two more elasticsearch nodes to relieve memory pressure (more
nodes means fewer indexes per nodes which requires less memory to
manage). And two more nodes gives us more disk to retain older indexes
in. These new nodes should allow us to retain at least 3 weeks of
indexed logs.
Change-Id: I3a5a02311e939c8147e401110c7b96d085eb8274
Reviewed-on: https://review.openstack.org/36305
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
Use mutliple discover nodes to determine elasticsearch cluster
membership. Put a timeout on recovery starting instead of the default
to recovery immediately. Describe cluster topology in elasticsearch yaml
config so that it can make smarter decisions. Round robin kibana
requests across each discover node.
Change-Id: I08ef9dd158ddf6a6ce01dfb2050626f543d45b10
Reviewed-on: https://review.openstack.org/34106
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Add logstash-worker4 and logstash-worker5.
Change-Id: Ibd1a618c4283b1d87904794d35963f69d1945b4e
Reviewed-on: https://review.openstack.org/33998
Reviewed-by: matthew wagoner <zxkuqyb@gmail.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
We need to expand our elasticsearch install base. Update puppet to make
this possible.
Change-Id: Id0dae839b12ebf47715cf40a363832e0f661a94f
Reviewed-on: https://review.openstack.org/33910
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
* manifests/site.pp: Add node name patterns for fedora18 production
and development slaves with Python 3 support switched on.
Change-Id: I7eac41e6dccde50be962cbc3fda21ac90c3d246b
Reviewed-on: https://review.openstack.org/34084
Reviewed-by: Dan Prince <dprince@redhat.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
This commit fixes bug 1182154
The maven workflow for versioning and deployment is at odds with how
we like to do it. For versioning, our convention is to get info
from SCM to set the project build versions. For deployments
to remote repositories we prefer to use curl instead of mvn deploy
due to maven security vulnerabilities. Our python builds
have already been setup to set package versions from SCM and
deploy to pypi using curl so this commit is to make
maven versioning and deployments similiar our python
versioning and deployment workflow.
This commit does the following:
Setup a project version string as an environement variable
so we can pass it to maven builds. The version string is
retrieved from information in git. This makes the build versioning
workflow similar to how we build python packages.
This setup expects a variable called '$project-version' in
the root 'version' element (i.e. <version>${project-version}</version>)
of the maven project's pom.xml file.
For general (throw away CI) builds we do the following:
1. generate a package 'myplugin.hpi' with version '1.3.0.4.a0bc21f'
in the MANIFEST.MF file. The '4' is the number of commits since
last tag.
2. publish 'myplugin-1.3.0.4.a0bc21f.hpi' to tarballs.o.o
For release builds we do the following:
1. generate a package 'myplugin.hpi' with version '1.3.1'
in the MANIFEST.MF file.
2. publish 'myplugin-1.3.1.hpi' to tarballs.o.o
3. publish 'myplugin-1.3.1.hpi' to repo.jenkins-ci.org
Passes the jenkins credentials from hiera to the pypi slave so
we can use it to deploy released plugins to repo.jenkins-ci.org
Creates a generic jenkinsci-upload job that will deploy
released jenkins plugin artifacts to a remote repository with
user credentials from hiera. It will use the same curl deployment
method as the pypi-upload job.
Change-Id: If1306523a28da94ee970d96b7a788ca116348de7
Reviewed-on: https://review.openstack.org/31875
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
This change reorgs the logstash log pushing so that there is a central
gearman server that listens to Jenkins ZMQ events which are then
converted to per log file gearman jobs which are processed by gearman
workers. The central gearman server will live on logstash.o.o and the
existing logstash-worker hosts will be converted to gearman log pusher
workers.
This commit includes relavent documentation changes.
Change-Id: I45f7185c2479c54b090d223408dff268e1e8d7db
Reviewed-on: https://review.openstack.org/32455
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
The openstack_project class needs to be included to make the
jenkins_ssh_key variable available to nodes.
Change-Id: I57fb6f24755bf123cf9f6f9b724035d0ababf8f3
Reviewed-on: https://review.openstack.org/31581
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
The openstack_project::slave refactor defaulted the Jenkins' SSH pub key
to an empty string. This erased the authorized keys file for the Jenkins
user on our specialized slaves. Restore that file's contents.
Change-Id: I28067017ec2dc36feef9df076099813abf820ee9
Reviewed-on: https://review.openstack.org/31419
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
Add an SSH keypair for releasestatus so that it can connect to
review.openstack.org to grab review data. Also add review.o.o
public key to known_hosts.
The data in hiera was already added.
Change-Id: I193dfad5b229a0c193ce35d5a8917b0b3b86c117
Reviewed-on: https://review.openstack.org/30881
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
Logstash performs filtering in a single thread so it does not scale up
very well. Work around this by scaling Logstash out to multiple indexer
hosts.
Current plan is to have a small (2GB) kibana web front end host that
does nothing but talk to elasticsearch, three 4GB logstash indexers that
will run a single log-pusher.py + logstash indexer with some partition
of the logfiles assigned to each indexer, and finally the existing large
elasticsearch node.
Eventually properly load balancing log processing across the worker
nodes would be great, but the current partition method should work well
enough with little additional effort.
Change-Id: Ifc6396560934314ffd6a7c47eb2acff9e9c2a7af
Reviewed-on: https://review.openstack.org/30573
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
