This patch adds jobs to build maven plugins and deploy them
to the maven nexus repo at https://oss.sonatype.org
Closes-Bug: #1082812
Change-Id: I283d475ab18819391f282234b063522abbf09eda
To avoid namespace issues, rename openstack_project::elasticsearch
to openstack_project::elasticsearch_node.
Change-Id: Icd215cd0c13f06d279a477d0885d33665ce125ae
* manifests/site.pp: Pass new mysql DB variables to
openstack::etherpad*.
* modules/etherpad_lite/manifests/apache.pp: Fix broken /etc/ssl/certs
permissions (0700 -> 0755).
* modules/etherpad_lite/manifests/init.pp: Update default nodejs and
etherpad versions. Remove ep_headings plugin install. New plugin define
should be used for this instead. Stop making the etherpad-lite ref to
checkout optional (defaults to develop). Note these changes are probably
not going to be backward compat.
* modules/etherpad_lite/manifests/plugin.pp: Define to install etherpad
lite plugins.
* modules/etherpad_lite/manifests/site.pp: Simplify DB support and
remove support for the dirty DB type.
* modules/etherpad_lite/templates/etherpad-lite_settings.json.erb: Bring
settings erb up to par with latest template.
* modules/etherpad_lite/templates/etherpadlite.vhost.erb: Update rewrite
rules for new etherpad. Instead of allowing nice pad urls rooted at /
redirect these url to /p/padname. Etherpad does not deal well with a
change in root path as /p/ is hardcoded in many places.
* modules/openstack_project/manifests/etherpad.pp
* modules/openstack_project/manifests/etherpad_dev.pp:
Update to use new etherpad module setup. MySQL DBs are now externally
managed, pass in needed connection info.
* modules/mysql_backup/manifests/backup_remote.pp: New define to backup
remote DB servers.
* modules/mysql_backup/templates/my.cnf.erb: Template for a my.cnf to be
used by the cron in backup_remote.pp. Allows for easy connectivity from
server using MySQL DB as root.
Change-Id: I1250297674b91e81d59cd28c07c52e09967ca548
Add jenkins job builder tripleo script (based on pbr).
Add prepare_node_tripleo.sh which will be called by nodepool.
Add prepare_tripleo.sh script, based on the devstack script
but pulling in just the tripleo repositories with pull-tools
tripleo-incubator script.
Add tripleo stanza to nodepool itself.
Update site.pp with variables for tripleo authentication.
Add experimental pipeline to zuul's layout.yaml for this job on
tripleo-incubator.
Change-Id: I03433d08585ba3a783fe939264d7521cafa83e41
Also, reduce number of ready nodes slightly to better observe how
nodepool reacts.
Allow nodepool to talk to zuul's gearman server.
Change-Id: I764482062525b6ca5454b53cbd56861b24fd3764
* manifests/site.pp: Expect the SSH key in /home rather than in
/var/run.
* modules/elastic_recheck/manifests/init.pp: Change the recheck
shell account for elastic-recheck to use a homedir in /home.
Change-Id: I43f4a8cdcb803328388b7e92732e5cb9d6acb2a8
This elastic recheck thing that classifies failures based on logstash
queries is awesome. We should run it on the OpenStack developer
infrastructure. Deploy it on logstash.o.o to keep related services
together.
Change-Id: I16ae7a32c2c48474454df17931779d0f60c3ed85
Add a groups-dev node to openstack infra, based on drupal
environment. This node checks out the repository:
https://git.openstack.org/openstack-infra/groups
Build a new deployment from scratch, including drush make
distbuild and drush si auto-installation features.
Change-Id: I71eef1c14724ceb13a896ff768923148dedcc2ef
This is the skeleton we'll fill out with any additional
functionality needed by the salt-trigger slave.
Change-Id: I928c2fa4c64e69140b673615a8c18e4eea79e952
This works by installing PyPy from the Ubuntu PPA, and then invoking
`tox -e pypy` (similar to all the other Python buidlers). PyPy shares
nodes with the Python3 builders.
Change-Id: Ibf683c1995b0f5c04c93c56010befb222b386d7a
* manifests/site.pp: Use distinct manifests for git.o.o and git.o.o
backends. This allows for the haproxy server to not serve git content
and purely be a load balancer.
* modules/cgit/manifests/init.pp: Remove haproxy from cgit module.
Remove stale xinetd cleanup. Select git daemon port when selecting
HTTP(S) ports.
* modules/openstack_project/manifests/git.pp: Make git.pp a manifest to
load balance git servers with haproxy.
* modules/openstack_project/manifests/git_backend.pp: New manifest to
manage servers that actually serve git content. They sit behind a load
balancer.
* modules/openstack_project/manifests/review.pp: Stop replicating repos
to git load balancer.
Change-Id: I343a0d1e0a7b93874c2e2299ed974a3304957efb
* install_modules.sh: Add puppetlabs-haproxy forge module.
* modules/cgit/manifests/init.pp: Add haproxy config to load balance
https, https and git protocol git access. Each git server will host git
http on port 8080, https on port 4443 and git protocol on 29418. These
endpoints will then be load balanced by a single haproxy instance
listening on ports 80, 443 and 9418. The use of haproxy and having
services listen on offset ports to accomodate haproxy is toggleable
using the $balance_git and $behind_proxy boolean flags.
Additionally, configure rsyslog for haproxy.
* modules/cgit/files/rsyslog.haproxy.conf: Enable syslog over UDP on
port 514. This is needed by haproxy to perform logging. Send local0
messages to /var/log/haproxy.log.
* modules/cgit/templates/ssl.conf.erb: Make Apache https listen port
configurable. Remove default virtualhost.
* modules/cgit/templates/httpd.conf.erb: Make Apache http listen port
configurable.
* modules/cgit/templates/git.vhost..erb: Make Apache http(s) listen
ports configuruable. Allow http without redirecting to https as a
fallback option to accomodate CentOS clients.
* modules/openstack_project/manifests/git.pp: Pass load balancer
variables through to the cgit manifest.
* manifests/site.pp: Configure git.o.o to run the load balancer haproxy
and balance across the new gitXX.o.o nodes.
Change-Id: Icefc5923cff9a7c6ce62c1923ec2ea87ebc6474a
The devstack cache steps were being run as the ubuntu user; make
sure they are run as jenkins.
Also update the nodepool configuration file to current values and
include the jenkins credential id.
Change-Id: Icbe8bee3b1800e28034cd36d4d328a6793d55b0e
The pubkey being used previous was the server pubkey, not the
gerrit2 user pubkey. The gerrit2 user pubkey is now in hiera
and switched git.openstack.org to use that for the
authorized_keys file.
Change-Id: I6665ef119071b8b68e4cb6bb7f7e8d79ad04fcbc
Logstash.o.o needs to be given the host and port for its elasticsearch
discover node list. Fix the change that used just a list of hosts.
Change-Id: If421007f633eb54a5dc65d5ede57d1e0a6f1c5d9
Proposal.slave.openstack.org is used to push change proposals into
Gerrit. To do this the slave needs to have the Jenkins' private SSH key
installed. Make it so.
Change-Id: Id3106d3167da0569370000e3c1a724a88abbd245
* manifests/site.pp: Put elasticsearch nodes in a list variable that is
reused.
* modules/logstash/templates/elasticsearch.yml.erb: Update cluster
settings to be more appropriate for the larger cluster size, including
changing the minimum number of master eligible nodes that must be seen
before becoming operational in a cluster. Should help prevent split
brained clusters.
* modules/openstack_project/manifests/cacti.pp: Add new node to cacti
node list.
Change-Id: I9a18422aa56a56b9ad030892a312772d9b6b2a79
The tx slave is going to be used for other jobs that propose changes to
Gerrit. Give it a more appropriate name. This lines up with the move
from hosting this under rackspace legacy to rackspace nova.
Change-Id: I7168d62a4a0aae27d2cd2a7dc6c37595604c8251
Now that Zuul and Jenkins use Gearman we don't need to push Jenkins auth
details down into the zuul manifests. Removes these parameters and vars.
Change-Id: I744977f002eb9fb60c3392f0281ab033ffe2bb12
This calculates the full LOG_PATH in the Zuul config and pases it
to Jenkins. The new path is similar to the old but with the
substitution of a short ZUUL_UUID at the end instead of the Jenkins
build number in order to avoid collisions from multiple Jenkins
masters.
Periodic jobs add a node name to their log path to avoid collisions
from multiple masters. Unfortunately, that value is not accessible
to the logstash worker. This can be solved by having Zuul trigger
periodic jobs.
Add the ZUUL_REF to logstash as "build_ref".
Requires https://review.openstack.org/#/c/39130/
Change-Id: I40bad59e3ad8ed6b4706762ed8b833fd15c13b0d
Add jenkins01 and jenkins02, both of which will have unit test and
devstack slaves. Leave jenkins.o.o alone; over time it will be
reduced so that it alone has special jobs and privileged slaves
attached to it.
Note that currently all of the jobs will be defined on all nodes,
including jobs on timers. I think the long-term fix for that is to
have zuul schedule timed jobs.
Change-Id: I10bbd5555e5194b1031700975d5b3ae6b458b8b3
We keep adding more tempest tests to run per patchset. These tests are
the most expensive to index as they come with many megabytes of
uncompressed logs. Add more logstash worker nodes in an attempt to keep
up.
Change-Id: I41d35792d58d1b0456927b0f5b1b7728bf3d145c
* manifests/site.pp: Unfortunately node names like precise3k-1
match the existing node name pattern precise-?\d+.* and end up with
the precise.slave.openstack.org shared puppet cert as a result.
Switch to precisepy3k which won't have this unintended side effect.
Change-Id: I2a530b0e19af769700e85f9060c2cbd8592eef1d
And use that when constructing log paths and URLs. Use a substr
of the change id or commit sha when constructing URLs so the log
directories are deeper.
Make copying the test results a macro (it's used in several places).
Update the gearman log client to take advantage of the new parameter.
Requires https://review.openstack.org/#/c/36304/
Change-Id: I64faa35eddc4105271efa3de4f83b608b77655c2
* manifests/site.pp: Add a node pattern for slave servers which run
Ubuntu Precise Pangolin and default to Python 3.3 for the Puppet PIP
package provider.
* launch/README: Starting with precise3k slaves, individual Puppet
certificates will now be used per-server.
Change-Id: I2cde440720afd8014fc98dfd133d29a428751a9d
* manifests/site.pp: List the two new elasticsearch nodes in the
appropriate lists.
* modules/logstash/manifests/elasticsearch.pp: Do not restart
elasticsearch when config files change. Service restarts are costly and
should be manually performed when necessary. Otherwise puppet should
simply update the config files.
* modules/logstash/templates/elasticsearch.yml.erb: Update elasticsearch
config with new cluster topology. Increase memory available for
indexing.
* modules/openstack_project/manifests/cacti.pp: Add new nodes to cacti
monitoring list.
Adding two more elasticsearch nodes to relieve memory pressure (more
nodes means fewer indexes per nodes which requires less memory to
manage). And two more nodes gives us more disk to retain older indexes
in. These new nodes should allow us to retain at least 3 weeks of
indexed logs.
Change-Id: I3a5a02311e939c8147e401110c7b96d085eb8274
Reviewed-on: https://review.openstack.org/36305
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
