440 Commits

Author SHA1 Message Date
Monty Taylor
51949062fb Add replication key for gerrit from hiera
The ssh key in ~gerrit2/.ssh/id_rsa which is what is used for outbound
ssh-based replication is currently just kinda there by hand. Add management
of the files there.

Change-Id: I5bfea4543d6eb46ba2e9f3c791f4e6b6c5534522
Closes-Bug: 1209464
2014-02-09 21:13:31 +00:00
Jenkins
dc13aa372f Merge "Redo slave manifests for clarity and correctness." 2014-02-05 22:27:38 +00:00
Jenkins
eccb3478f9 Merge "point zuul-dev to review-dev" 2014-01-31 22:44:06 +00:00
Jeremy Stanley
f20639a2be Install and use the Gerrit welcome-message keypair
Change-Id: I7c23112664e43713f8dfc4bd0e4b9aec46b71500
2014-01-31 22:04:15 +00:00
Clark Boylan
0f4b0fae64 Redo slave manifests for clarity and correctness.
The differences between openstack_project::slave,
openstack_project::slave_template, and openstack_project::bare_slave
were not always clear. Keep openstack_project::slave as the default long
running slave manifest, but replace slave_template with a
single_use_slave.pp to make it clear where single use slave config
begins. Add the ability to toggle automatic upgrades and jenkins sudo
rights to this new manifest. Finally, add a more verbose comment to
bare_slave explaining what it is useful for (having a jenkins like slave
host that doesn't need a firewall or ntp or automatic upgrades).

Change-Id: I3989c9e6ad9469f441ca5d3627f7b3b704d8a8da
2014-01-30 10:37:36 -08:00
Khai Do
3e4ef9e003 point zuul-dev to review-dev
Partial-Bug: #1082781
Change-Id: I9fab6a763c45cfa0b7c2970f98876d178df16690
2014-01-29 15:47:56 -08:00
Ruslan Kamaldinov
265bd8fe1d Add Storyboard puppet module
There are two major parts being installed with this module:
1. storyboard-api - REST API service served  with
   apache mod_wsgi module
2. storyboard-webclient - static html/css/js files.
   This project is built and published to tarballs.o.o,
   from where it'll be installed with this puppet module

This module requires three configs from Hiera:
* storyboard_db_host
* storyboard_db_password
* storyboard_db_user

Installed projects:
* http://git.openstack.org/cgit/openstack-infra/storyboard/
* http://git.openstack.org/cgit/openstack-infra/storyboard-webclient/

Things to be added in later commits:
* Documentation for ci.openstack.org
* Configure logging (once supported by storyboard)
* SSL

Change-Id: If3da06f8d20a6282036f1f9f063c25a6d0db60c6
2014-01-29 06:47:37 +04:00
Monty Taylor
2dfcf4fb71 Add three new jenkins servers
Change-Id: I1143c1c7446890c47be12d8b115560bddadc1919
2014-01-22 12:50:40 -05:00
Jenkins
18f4498ec1 Merge "Set up openstackid module" 2014-01-14 22:46:47 +00:00
James E. Blair
0c8845494d Don't have zuul replicate to git.o.o
Change-Id: I8a80b26d843501d0e426bcff84ad8f71c5a68fc5
2014-01-08 20:24:51 +08:00
James E. Blair
62e37a37d8 Fix passing zuul public key to git backends
Change-Id: I774d36ce82266e1431ff89acdc6450dbb1626004
2014-01-08 16:50:59 +08:00
Jeremy Stanley
73c0175819 Allow zuul to push to git servers
* manifests/site.pp
* modules/openstack_project/manifests/git_backend.pp: Add the zuul
server's public SSH key to a dedicated zuul account on the git
server farm. This will allow zuul to push arbitrary references into
its scratch repositories. Also set ownership on its scratch
repository hierarchy and pack them, properly skipping it in the
existing cgit repo repack since ownership differs.

Change-Id: I4b5b583847c828aadc903270433882531e5b1ea7
2014-01-08 16:43:36 +08:00
James E. Blair
14694564df Fix typo in puppet
Change-Id: I39c29152e985aceefd1cc9032ca51081668d787d
2014-01-08 16:12:39 +08:00
James E. Blair
2866945db2 Have Zuul replicate to git.o.o
Change-Id: I750e8ea75181dacb10275a38fa702ed1edb56d4c
2014-01-08 15:52:00 +08:00
James E. Blair
f6500357f0 Add jenkins03 and 04 to most places
Add them to all the places it's safe to do so for now.  Don't actually
spin up any nodes for them yet.

Change-Id: I59e97be7e5b094af3153bc7d5dce0cff57996f55
2014-01-07 09:58:17 +08:00
James E. Blair
2bb28339cb Move elastic-recheck bot to status.o.o
Also, normalize some parameters to the status manifest to make it
more readable (since they follow the same pattern).  And make the
ssh config for elastic-recheck bot match reviewday.

Change-Id: I2417f121e7b3685aab9540504cdd4c6db1754e67
2013-12-31 19:35:47 +00:00
Jeremy Stanley
318c79b9fc Set up openstackid module
Refactor the openstack_project::openstackid_dev module out into a
top-level openstackid module in preparation for multiple servers,
set up Apache to serve content out of /srv/openstackid, add an
/etc/openstackid/database.php file with connection details injected
from hiera and keep an updated clone of openstack-infra/openstackid
in /opt/openstackid.

Change-Id: Icdde594384e3af27c8dd185a51b9e5a71619fb7b
2013-12-27 22:56:55 +00:00
James E. Blair
90aa64a644 Remove status.o.o vhost from static.o.o
Change-Id: I8e7ee035b10895043622041bf8bdfef84162ffd5
2013-12-19 14:57:02 -08:00
James E. Blair
757d7fa5bf Remove old logstash worker nodes
Change-Id: If67bf7ce363a8db500fc893dc38f7f4f46bda3f5
2013-12-18 13:34:26 -08:00
Jenkins
dd37040caa Merge "Increase to 16 logstash worker nodes" 2013-12-18 18:28:59 +00:00
James E. Blair
b08ccad2cb Increase to 16 logstash worker nodes
Add '%02d' versions of the system names so that when we replace
the current workers with performance nodes, they will have more
scalable names.

Change-Id: I6c9d8aa6338bdd94649096a13f2d3dbeb1daec08
2013-12-18 09:58:54 -08:00
James E. Blair
a750e4aa5b Add status.openstack.org vm
Let's branch status.o.o off from static.o.o into its own machine.
For now, keep the extra /static/ in paths to ease the transition
(so we can use the same files and templates).  We can remove them
later if desired.

Change-Id: If38197db79b9f6adfcdcf40f10d4797c7df1c620
2013-12-11 13:17:21 -08:00
Jenkins
c3dbd35c46 Merge "Review and create Puppet configuration for dev server" 2013-12-09 19:48:28 +00:00
Jenkins
dac9a30496 Merge "add nodepool to jenkins-dev server" 2013-12-07 01:44:40 +00:00
Jenkins
97322c888c Merge "new maven plugin jobs to build and deploy maven plugins to maven nexus" 2013-12-06 21:54:56 +00:00
Sean Dague
b3af619900 move to elastic-recheck web console in share
this provides the changes needed to handle the new elastic-recheck
web console installed from the elastic-recheck package in the
share tree.

factor out bot code to elastic_recheck::bot

this change moves elastic_recheck bot starting code out into
a separate module, which lets us include all the elastic_recheck
base module into the static site config.

Change-Id: If53523754494a746c152c5d1384db5426b096fc1
2013-12-06 15:52:40 -05:00
smarcet
56472aa9d4 Review and create Puppet configuration for dev server
blueprint sso-openid-provider

Change-Id: I7966f59df6bd6ba851b507c7a408f076d7031abe
2013-12-03 17:34:02 -03:00
Khai Do
efe3323255 add nodepool to jenkins-dev server
We are co-locating nodepool on jenkins-dev to allow us to test
jenkins upgrades.  This dev nodepool is configured to only spin
up machines in one AZ on hpcloud.

Change-Id: I73f6d4373fa79fec8845d7d81fa1235f7fe15482
2013-12-03 18:21:31 +00:00
Khai Do
9aef0e90d8 new maven plugin jobs to build and deploy maven plugins to maven nexus
This patch adds jobs to build maven plugins and deploy them
to the maven nexus repo at https://oss.sonatype.org

Closes-Bug: #1082812

Change-Id: I283d475ab18819391f282234b063522abbf09eda
2013-12-02 17:08:49 +00:00
Arx Cruz
5e5023ebae Zuul puppet support to zuul_url option
This adds the zuul_url support in puppet.

Change-Id: Ie37744f11452efc4029c1fe93d880dd3a4ecb831
2013-11-27 09:58:47 -02:00
James E. Blair
b17d73670f Create jenkins03 and jenkins04
Change-Id: I8db56b9a7a2ad8f423d052c07d4908480d9f8762
2013-11-20 10:56:02 -08:00
Monty Taylor
d1c3803dcb Remove old pypi host cruft
We no longer use this, so keeping it in puppet is confusing.

Change-Id: I8a5d2a0067063ba762d553f397079deef43fc43f
2013-11-12 11:34:08 -05:00
Ryan Lane
2f4c686edc Rename openstack_project::elasticsearch
To avoid namespace issues, rename openstack_project::elasticsearch
to openstack_project::elasticsearch_node.

Change-Id: Icd215cd0c13f06d279a477d0885d33665ce125ae
2013-10-18 16:22:13 -07:00
Jenkins
98aff895ee Merge "Update etherpad and etherpad puppet manifests." 2013-10-13 16:29:27 +00:00
Clark Boylan
c1ffb9f145 Update etherpad and etherpad puppet manifests.
* manifests/site.pp: Pass new mysql DB variables to
openstack::etherpad*.

* modules/etherpad_lite/manifests/apache.pp: Fix broken /etc/ssl/certs
permissions (0700 -> 0755).

* modules/etherpad_lite/manifests/init.pp: Update default nodejs and
etherpad versions. Remove ep_headings plugin install. New plugin define
should be used for this instead. Stop making the etherpad-lite ref to
checkout optional (defaults to develop). Note these changes are probably
not going to be backward compat.

* modules/etherpad_lite/manifests/plugin.pp: Define to install etherpad
lite plugins.

* modules/etherpad_lite/manifests/site.pp: Simplify DB support and
remove support for the dirty DB type.

* modules/etherpad_lite/templates/etherpad-lite_settings.json.erb: Bring
settings erb up to par with latest template.

* modules/etherpad_lite/templates/etherpadlite.vhost.erb: Update rewrite
rules for new etherpad. Instead of allowing nice pad urls rooted at /
redirect these url to /p/padname. Etherpad does not deal well with a
change in root path as /p/ is hardcoded in many places.

* modules/openstack_project/manifests/etherpad.pp
* modules/openstack_project/manifests/etherpad_dev.pp:
Update to use new etherpad module setup. MySQL DBs are now externally
managed, pass in needed connection info.

* modules/mysql_backup/manifests/backup_remote.pp: New define to backup
remote DB servers.

* modules/mysql_backup/templates/my.cnf.erb: Template for a my.cnf to be
used by the cron in backup_remote.pp. Allows for easy connectivity from
server using MySQL DB as root.

Change-Id: I1250297674b91e81d59cd28c07c52e09967ca548
2013-10-09 16:53:06 -07:00
Monty Taylor
647a446d8f Remove launchpad_sync module
We do not sync from Launchpad any longer.

Change-Id: I414436278953347b65dfb5bfc394d067fbff307d
2013-10-09 11:55:05 -04:00
Elizabeth Krumbach Joseph
6fd82112e8 Add initial files for tripleo in the gate.
Add jenkins job builder tripleo script (based on pbr).

Add prepare_node_tripleo.sh which will be called by nodepool.

Add prepare_tripleo.sh script, based on the devstack script
but pulling in just the tripleo repositories with pull-tools
tripleo-incubator script.

Add tripleo stanza to nodepool itself.

Update site.pp with variables for tripleo authentication.

Add experimental pipeline to zuul's layout.yaml for this job on
tripleo-incubator.

Change-Id: I03433d08585ba3a783fe939264d7521cafa83e41
2013-10-07 12:50:21 -07:00
James E. Blair
eefbb633b4 Start using gearman-based bursting with nodepool
Also, reduce number of ready nodes slightly to better observe how
nodepool reacts.

Allow nodepool to talk to zuul's gearman server.

Change-Id: I764482062525b6ca5454b53cbd56861b24fd3764
2013-10-05 08:26:20 -07:00
Jeremy Stanley
fd832de552 Move elastic-recheck shell account out of /var/run
* manifests/site.pp: Expect the SSH key in /home rather than in
/var/run.

* modules/elastic_recheck/manifests/init.pp: Change the recheck
shell account for elastic-recheck to use a homedir in /home.

Change-Id: I43f4a8cdcb803328388b7e92732e5cb9d6acb2a8
2013-09-30 21:44:03 +00:00
Clark Boylan
e0427cfbb7 Deploy elastic-recheck on logstash.openstack.org.
This elastic recheck thing that classifies failures based on logstash
queries is awesome. We should run it on the OpenStack developer
infrastructure. Deploy it on logstash.o.o to keep related services
together.

Change-Id: I16ae7a32c2c48474454df17931779d0f60c3ed85
2013-09-30 18:56:23 +00:00
Jenkins
2f81032060 Merge "adding staging server for activity board" 2013-09-20 21:20:52 +00:00
Marton Kiss
648a8af52f Add groups-dev.openstack.org node
Add a groups-dev node to openstack infra, based on drupal
environment. This node checks out the repository:
https://git.openstack.org/openstack-infra/groups

Build a new deployment from scratch, including drush make
distbuild and drush si auto-installation features.

Change-Id: I71eef1c14724ceb13a896ff768923148dedcc2ef
2013-09-17 11:43:00 +02:00
Jeremy Stanley
ef399b91af Add a Jenkins slave for triggering salt commands
This is the skeleton we'll fill out with any additional
functionality needed by the salt-trigger slave.

Change-Id: I928c2fa4c64e69140b673615a8c18e4eea79e952
2013-09-12 17:38:19 +00:00
Stefano Maffulli
93490ce25d adding staging server for activity board
Change-Id: Ib2b0de6e2cdc59f5c9dca90117d333f70dbab040
2013-09-12 14:40:44 +10:00
Alex Gaynor
32db246dd3 Add support for running tox with PyPy and use it for Marconi
This works by installing PyPy from the Ubuntu PPA, and then invoking
`tox -e pypy` (similar to all the other Python buidlers). PyPy shares
nodes with the Python3 builders.

Change-Id: Ibf683c1995b0f5c04c93c56010befb222b386d7a
2013-08-27 13:44:10 -07:00
Clark Boylan
f22f8f5aa1 Split git.o.o haproxy out of cgit module.
* manifests/site.pp: Use distinct manifests for git.o.o and git.o.o
backends. This allows for the haproxy server to not serve git content
and purely be a load balancer.

* modules/cgit/manifests/init.pp: Remove haproxy from cgit module.
Remove stale xinetd cleanup. Select git daemon port when selecting
HTTP(S) ports.

* modules/openstack_project/manifests/git.pp: Make git.pp a manifest to
load balance git servers with haproxy.

* modules/openstack_project/manifests/git_backend.pp: New manifest to
manage servers that actually serve git content. They sit behind a load
balancer.

* modules/openstack_project/manifests/review.pp: Stop replicating repos
to git load balancer.

Change-Id: I343a0d1e0a7b93874c2e2299ed974a3304957efb
2013-08-26 12:05:26 -07:00
Clark Boylan
dea75ed695 Load balance git requests.
* install_modules.sh: Add puppetlabs-haproxy forge module.

* modules/cgit/manifests/init.pp: Add haproxy config to load balance
https, https and git protocol git access. Each git server will host git
http on port 8080, https on port 4443 and git protocol on 29418. These
endpoints will then be load balanced by a single haproxy instance
listening on ports 80, 443 and 9418. The use of haproxy and having
services listen on offset ports to accomodate haproxy is toggleable
using the $balance_git and $behind_proxy boolean flags.
Additionally, configure rsyslog for haproxy.

* modules/cgit/files/rsyslog.haproxy.conf: Enable syslog over UDP on
port 514. This is needed by haproxy to perform logging. Send local0
messages to /var/log/haproxy.log.

* modules/cgit/templates/ssl.conf.erb: Make Apache https listen port
configurable. Remove default virtualhost.

* modules/cgit/templates/httpd.conf.erb: Make Apache http listen port
configurable.

* modules/cgit/templates/git.vhost..erb: Make Apache http(s) listen
ports configuruable. Allow http without redirecting to https as a
fallback option to accomodate CentOS clients.

* modules/openstack_project/manifests/git.pp: Pass load balancer
variables through to the cgit manifest.

* manifests/site.pp: Configure git.o.o to run the load balancer haproxy
and balance across the new gitXX.o.o nodes.

Change-Id: Icefc5923cff9a7c6ce62c1923ec2ea87ebc6474a
2013-08-22 10:52:04 -07:00
James E. Blair
86b0f3bfd5 Fix nodepool setup scripts
The devstack cache steps were being run as the ubuntu user; make
sure they are run as jenkins.

Also update the nodepool configuration file to current values and
include the jenkins credential id.

Change-Id: Icbe8bee3b1800e28034cd36d4d328a6793d55b0e
2013-08-20 19:58:38 -07:00
James E. Blair
a38e256eb2 Remove devstack launch nodes and jobs
Change-Id: Icc6ba916df45b2a763f85a73412f9c8e1df88701
2013-08-18 08:43:50 -07:00
James E. Blair
8222c6e00f Add nodepool host
Change-Id: Ib216bd400aa269ccdb17848b3870ab65400cf3d2
2013-08-16 13:15:42 -07:00