117 Commits

Author SHA1 Message Date
Monty Taylor
ad574bedee Changes to get nodepool working with trusty
There are few things that are different for our slaves on trusty. Of
note, we don't want to pin puppet, and we need to provide a version
for postgres.

Change-Id: Ibee78cd4fbeef2e6af43379d2bc3a0f0e9767a06
2014-05-28 10:09:35 -07:00
K Jonathan Harker
541fac117b Move jenkins-sudo bits to openstack_project
The jenkins-sudo privileges are specific to the slave_scripts directory,
which has already been moved to the openstack_project. Let's keep these
two things together.

Change-Id: I0ef16850bf97d54f1ac92e41e5bd31e7f6f40ea9
2014-05-23 11:55:20 -07:00
K Jonathan Harker
8e7f9e3458 Move openstack-specific config out of ::jenkins
The ::jenkins::slave class contained a lot of openstack-specific
configuration rather than configuration of a generic jenkins slave.

The term "bare slave" is overloaded and confusing: create simple_slave
and thick_slave to differentiate between the two meanings of "bare".
Some portions of ::jenkins::slave will move to simple_slave, some
portions to thick_slave, and some portions to slave_common (all in the
openstack_project module).

Change-Id: I5281a03a7f6da3f98714bcc59ae840ace8435578
2014-05-20 14:39:51 -07:00
Monty Taylor
d0061640be Remove accumulated python cruft
We nolonger need or want setuptools_git installed anywhere.
get-pip no longer needs ez_setup.py first. Also, github broke
redirection on Fedora, so change the URL.

Change-Id: I16d64695bf05e672fdc12236424b62e5cd5e5dc7
2014-04-29 10:10:01 -07:00
Jenkins
415842d361 Merge "Add a slave-script to Jenkins for pushing to swift" 2014-04-23 03:10:48 +00:00
Joshua Hesketh
7b4cfd7dd1 Add a slave-script to Jenkins for pushing to swift
I042cdd2dd2407f381cafcabc5c6b83d9b9a9eb00 adds instructions to
workers on how to send logs to swift using the FormPost middleware.
This slave-script allows you to pass in a directory to push to
swift using these instructions and returns a URL where they are
available.

Change-Id: I7ccafa408a88ef80317b1c546b3451c0033dce61
2014-04-01 14:01:36 +11:00
Akihiro Motoki
e18cb36161 Add language spepcific font package to Jenkins slave
Localized documents start to be generated in gate jobs and
they require language specific font packages. This commit
add language_fonts_packages parameter to accept language fonts.

It adds Japanese and Korean font packages to Ubuntu build
slave because gate jobs are now run on Ubuntu based hosts.
Pakcages for RedHat flavor will be added in future if needed.

Change-Id: Id4a0363e119f4458c270fb32bd428ebcfcc75225
Closes-Bug: #1278327
2014-04-01 03:15:43 +09:00
Jeremy Stanley
044ded07b4 Pin rake gem to 10.1.1 so it works with ruby 1.8.x
Without this, we get "Error installing puppetlabs_spec_helper: rake
requires Ruby version >= 1.9."

Change-Id: I81a32380cee1a35a50e1c3baf2e1afcd69b1614f
2014-03-25 12:57:22 +00:00
Jenkins
da0d8af6ee Merge "Set up opportunistic bare-metal postgres db" 2014-03-17 17:15:58 +00:00
Robert Collins
2172d4bfd2 Add haveged to jenkins slaves.
We're running headless servers - exactly the situation haveged is
designed for.

Change-Id: I5616f5926aea9a573d1bf1d617f1b5b8df672917
2014-03-15 16:23:17 +13:00
Jenkins
836493b0d0 Merge "Explicitly install gettext on job workers" 2014-03-10 21:17:15 +00:00
Jenkins
b9c70fb590 Merge "Use a sudoers file for jenkins sudo rights" 2014-03-10 15:56:48 +00:00
Jeremy Stanley
bfabbaf6ef Explicitly install gettext on job workers
Some providers include gettext preinstalled by default on their base
images, but others do not. Make this consistent by explicitly
installing it when not already present. Rackspace seems to have
recently ceased including it on their Ubuntu Precise images, which
precipitated this patch.

Change-Id: Idb86c805e2fc56d885081876642bd9d3dbf58434
2014-03-10 13:49:14 +00:00
Jenkins
afe0b82d28 Merge "Install memcached on slaves" 2014-03-06 23:15:50 +00:00
Khai Do
bff86de71e Fix update java alternatives on redhat systems
puppet was throwing the following error..
  nodepool.image.build.rax-iad.bare-centos6: Could not find dependent
  Exec[update-java-alternatives] for Package[ant] at |
  /root/config/modules/jenkins/manifests/slave.pp:86

because update-java-alternatives was only defined for debian, not for redhat systems.
This change adds java alternatives update to redhat.  Needed to change
'java-1.7.0-openjdk' to 'java-1.7.0-openjdk-devel' because 'java-1.7.0-openjdk' does
not install the javac compiler.

We are setting alternatives for debian and centos but not for fedora because the java 6
packages are not available on fedora.

Change-Id: I04b3e8292d3f31cd56ada9be99da8a234e13585a
2014-03-03 20:56:35 -08:00
Julien Danjou
7c178f129a Install memcached on slaves
This is needed to run Tooz integration tests.

Change-Id: Ibaa2c4adf81a50465910572a6b6d145640476a51
2014-02-28 17:11:12 +01:00
Jenkins
880924a621 Merge "Optionally give mysql user all global privs." 2014-02-27 22:13:23 +00:00
Jenkins
bf96db40f3 Merge "Change mysql package for Fedora >= 19 for bare-precise" 2014-02-27 19:24:49 +00:00
Clark Boylan
54ad9496cd Optionally give mysql user all global privs.
One our single use unittest slaves we want to allow unittests to create
and destroy arbitrary mysql database schemas. The simplest way to do
this is to grant all priveleges to the test user on the mysql server
globally. This is safe because the nodes are thrown away post test.

Change-Id: Ic92ff2f9c752d955e6f82eb4b88330b5448cfa75
2014-02-26 11:48:59 -08:00
Joshua Hesketh
8d9ebffe10 Set up opportunistic bare-metal postgres db
Currently only MySQL is being opportunistically tested for bare-metal
migrations. This change configures the Jenkins slaves to have a
postgresql database openstack_baremetal_ci.

Change-Id: Ifb2eb7b1ef15b28a326a7d7eb1f99dc675cc40b6
2014-02-26 16:09:08 +11:00
James Polley
d91718654c Make sure gawk is installed
extract_docs.awk (used to extract devtest.rst from devtest.sh, and
similiarly with other scripts that have docs auto-generated) does not
work in mawk, which is the default in precise.

gawk is already the default choice in Fedora 19, and seems to be the
default in later versions of Ubuntu. Haven't tested Debian.

Change-Id: I00cf5137d8e39c78a5d52b2cf094365a56deb353
2014-02-24 18:56:36 +11:00
Khai Do
4b1e84a4a3 update java alternatives to java 7
ant dependency requires java 6 so alternatives was set to java 6 on an ant package
install.  This change updates alternatives to point to java 7 after all packages
have been installed.

Change-Id: I7b8c975879dda085102a1d4740777d3d8c5f54ea
2014-02-21 15:51:05 -08:00
Jenkins
6ff82384d7 Merge "Basic gating and release jobs of openstackid project" 2014-02-20 19:19:35 +00:00
Marton Kiss
f46639efd6 Basic gating and release jobs of openstackid project
Add check, gate, post and release jobs to PHP/Laravel based openstackid
project. Extend precise node with php5-mcrypt package required for
build process.

Change-Id: If7858412d5c3425aab20c0d11a92cab4c86f47dd
2014-02-20 19:38:14 +01:00
Jenkins
c945537bd6 Merge "Setup tooling and jobs for openstack Gerrit ver 2.8 and beyond" 2014-02-20 18:22:34 +00:00
Khai Do
c3e2997ef8 Setup tooling and jobs for openstack Gerrit ver 2.8 and beyond
Previous versions of Gerrit used maven as the build tool, it has since
switched to use buck. I believe the switch happened in ver 2.7.  We are
now hosting a fork of Gerrit ver 2.8 so we need to setup tooling and
jobs to build newer versions of gerrit.  This patch sets up tooling and
jobs required for building the openstack Gerrit ver 2.8.  We now have
two sets of jenkins jobs to build gerrit, one for Gerrit prior to ver
2.4.x (maven) and one for Gerrit post ver 2.7 (buck).

Notes:
1. Buck does not have OS packages for easy installation so it needs to
be built from source using ant (requires gcc as well).
2. Maven jobs execute only when commits are made to 2.4.x branch.
3. Buck jobs execute when commits are made to branches other than 2.4.x
4. The origin post job 'gerrit-package' now publishes artifacts to
tarballs.o.o/ci/gerrit instead of tarballs.o.o/ci
5. Buck requires the zip package to build gerrit.
6. Buck and gerrit clones need to be at same directory level

Partial-Bug: #1082781
Change-Id: I1de9f127fa6826fb0a686d12dc77c0edff34be7c
2014-02-14 11:44:08 -08:00
Jenkins
eba2144fe1 Merge "Update groups gating scripts" 2014-02-13 11:52:31 +00:00
James E. Blair
200ce362a2 Use a sudoers file for jenkins sudo rights
This way we can start with the file in place on all single-use
slaves, and then remove the file at the beginning of jobs that
should not be able to sudo (for example, unit test jobs).

Change-Id: I37aabdba89d00b45365126c8f776ae6ef8357c8f
2014-02-12 08:02:40 -08:00
Marton Kiss
15761d181a Update groups gating scripts
Community portal build process failed due php package deployment
permission errors. This patch add php5-cli package to jenkins slave
and relocate building script from groups project to groups.yaml file.

Change-Id: I462be6bf1ea4e647e262f15427f74fab36e9c5d6
Closes-Bug: 1278344
2014-02-12 15:23:38 +01:00
Jeremy Stanley
6a069b086d Correct MySQL DB priv names for citest grants
The old configuration was resulting in a puppet error of...

    err: /Stage[main]/Jenkins::Slave/Database_grant
        [openstack_citest@localhost]/privileges:
        change from  to create, drop failed: The privileges
        'create,drop' are not valid. Did you mean
        'create_priv,drop_priv'?

Change-Id: I325cabe2fce8a6893b03d9deb106ed2ae2a29364
2014-02-11 23:48:24 +00:00
Clark Boylan
0e94117fa4 Give test mysql user create and drop perms.
Give the openstackci_test Jenkins slave test user permissions to CREATE
and DROP databases. This allows parallel tests to create as many DB
schemas as necessary to run tests without locking in the tests
themselves.

Change-Id: I0466ac8f318b7dc397fac6c16553e284bdb7d707
2014-02-10 14:16:46 -08:00
Clark Boylan
9833b3c703 Make postgres work on Jenkins slaves again.
A few upstream puppetlabs postgres module bugs need to be worked around
in order to have postgres servers installed on our slaves. Need to
create the postgres user and group before the upstream module runs and
provide hostnames to the postgres server config instead of ip addresses.

Change-Id: I68cea595d0cd6ad9b2155166ff473d1375e9b7ac
2014-02-06 13:35:21 -08:00
Arx Cruz
77966dcaf5 Change mysql package for Fedora >= 19 for bare-precise
For some reason, puppetlabs-mysql used by openstack-infra
only checks if the osfamily is RedHat and set the package
name to mysql-server (which doesn't exist in fedora).
I didn't check new puppetlabs-mysql releases (openstack
is using 0.6.1 and latest one is 2.1). Perhaps in future
this patch will no longer be necessary.

Change-Id: I29fb931f1fe70dcfe46dad71a4822a75b001ca5d
2014-02-05 01:32:13 -02:00
Jenkins
3abecd6567 Merge "Pull in the zookeeperd package instead of the zookeeper package" 2014-01-30 00:41:12 +00:00
Joshua Harlow
90e6b63d8b Pull in the zookeeperd package instead of the zookeeper package
Instead of installing the zookeeper package (which is just the basic
files) we should just install the zookeeperd package (which includes
the upstart scripts to run the zookeeper server) so that those who
want to test against zookeeper do not have to start/stop the server.

Change-Id: I83ee1c30fb8adb345f36c5facf3f06dcf059ef0f
2014-01-29 16:24:40 -08:00
Jeremy Stanley
7b029c87d1 Temporarily pin tox<=1.6.1
The release of tox 1.7.0 happened today, and we're clearly not ready
for it yet. Stop the bleeding by sticking with 1.6.1 while we try to
sort out the issues.

Change-Id: I3289828ea9a0edadcf40c06212b0925957b99f73
Related-Bug: #1274135
2014-01-29 14:34:03 +00:00
Jeremy Stanley
7282ca4539 Really remove Ubuntu Cloud Archive for now
* modules/jenkins/manifests/slave.pp: Make sure the sources.list.d
file for UCA left behind by the revert of bdcc115 gets cleaned up so
we don't accidentally install new packages from it.

Change-Id: Icf3243b08f0ab3904b29031857db01f735d0f410
2014-01-07 14:42:16 +00:00
Jenkins
a3168ee32c Merge "Add libvirt-dev in anticipation of python-libvirt" 2014-01-07 05:00:42 +00:00
Monty Taylor
0385b968ef Add libvirt-dev in anticipation of python-libvirt
python-libvirt now exists on PyPI. In anticipation of wanting to use it,
we need to ensure that libvirt-dev is installed on our slaves, so that
pip install python-libvirt will work.

Change-Id: I5958de644188e2745417a3cda319a2ea03896e7f
2014-01-01 09:35:21 -06:00
Arx Cruz
5048a68fc5 Alter zookeeper params for Fedora.
The zookeeper package for Fedora is zookeeper, instead of
zookeeper-server. This patch fix this problem when you
try to apply puppet jenkins slave on Fedora

Change-Id: I868987fa7f8c989e27444a6241a57f62372e645d
2013-12-17 14:36:39 -02:00
Jeremy Stanley
265494334d No zookeeper packages on CentOS slaves
Change 12c4749a tried to install zookeeper on all Ubuntu and CentOS
unit test slaves, but there is not actually a zookeeper-server
package in CentOS or EPEL. Install only on Ubuntu instead.

Change-Id: Ia50f016ef8d53dada63dd3218c7d731016b94383
2013-12-16 16:09:06 +00:00
Jenkins
ac4d4a1f6f Merge "Install libtidy on slaves" 2013-12-11 00:21:33 +00:00
Jenkins
5e097fa8d4 Merge "Install ZooKeeper on slaves" 2013-12-10 00:36:01 +00:00
Julien Danjou
12baff87ba Install libtidy on slaves
This is required by Python tidy module, that is a dependency of
sphinxcontrib-docbookrestapi. The later is used in at least Ceilometer
to build the documentation.

Change-Id: Ib9d9a14a8169ef5d7d3c77fd711515237ae5b0e7
2013-12-09 17:52:39 +01:00
Julien Danjou
12c4749ad5 Install ZooKeeper on slaves
Change-Id: Id29ede88a579472079877542293592553ebe09d5
2013-12-06 14:41:23 +01:00
Jeremy Stanley
33a62b6f61 Use force when purging
* modules/asterisk/manifests/init.pp
* modules/jenkins/manifests/master.pp
* modules/jenkins/manifests/slave.pp
* modules/openstack_project/manifests/gerrit.pp
* modules/openstack_project/manifests/jenkins.pp
* modules/openstack_project/manifests/nodepool.pp
* modules/openstack_project/manifests/static.pp: When a directory is
puppet-managed for content using recurse, replace and purge you also
need force or empty subdirectories will fail to be removed. What's
worse, subscribing to that directory will cause a refresh to be
triggered for it on every agent run.

Change-Id: I232d6ba98475522f391f469c194a4450c7a0b2e1
2013-12-05 18:38:04 +00:00
Jenkins
b38aaa6293 Merge "Install requirements repo on slaves" 2013-12-04 16:30:52 +00:00
Clark Boylan
753e337fcc Use purge option where appropriate.
* modules/jenkins/manifests/master.pp
* modules/jenkins/manifests/slave.pp: Recursively purge the
slave_scripts dir. This will remove old slave scripts.

* modules/openstack_project/manifests/gerrit.pp: Recursively purge the
gerrit ACLs dir. This will remove old and renamed ACL files.

* modules/openstack_project/manifests/nodepool.pp: Recursively purge the
nodepool scripts dir. This will remove old and renamed nodepool scripts.

* modules/openstack_project/manifests/static.pp: Recursively purge the
devstack log help files. This will remove old and renamed help files.

Change-Id: If739b274075781dcd0d0836bca96dd1764c6ee19
2013-12-02 14:15:51 -08:00
Monty Taylor
4643f50921 Install requirements repo on slaves
In advance of starting to use the requirements project as the basis
for doing mirror selection, ensure that the repo is installed on all
of the jenkins slaves.

Change-Id: I893dd82e759f6a84a62400cdcbc7596b19ec5923
2013-11-29 09:17:58 -05:00
Clark Boylan
63f0890579 Consolidate rsyslog configs.
Puppet doesn't allow non unique namevars. Having mutliple rsyslog
service definitions violates this rule. Work around this by
consolidating all rsyslog configs in openstack_project::template.
In the process of doing this it became apparent that the ensure whoopsie
was absent resource should be consolidated in to template.pp as well so
this was done as part of the larger consolidation.

Change-Id: I7effed7bacb85386f0cf36edffe442e2cad944d0
2013-11-22 14:08:10 -08:00