524 Commits

Author SHA1 Message Date
Monty Taylor
290d7f7435 Make the fallback sysadmins value the same
The fallback value for sysadmins was was admins in some places
and admin in others. However, re: jeblair:

The fallback value for sysadmins should be "[]", which is actually the
default value. Configuring the mail system to send root's mail to an
account called 'admin' which may or may not exist and if it does exist
may send mail to root is not a good idea.

The default of [] will deliver root's mail normally.

Change-Id: Iaeca86ef135b6a3210541618d48caf4058dc7966
2014-07-13 08:53:25 -07:00
Marton Kiss
bd9609e748 Add groups.openstack.org instance
site.pp and groups.pp update for groups portal. To make it alive
we need to do the following preliminary tasks:
- create remote database in trove for groups
- add hiera variables similar to groups-dev
- launch the instance with those manifests

Change-Id: I91d7b35ad9aabc2237d5516918fa6ce8d92ee5af
2014-07-09 20:11:39 +00:00
Monty Taylor
c8a29bea6d Delete the old cron-based puppet thing
Also, delete the legacy non-nodepool slave references.

Change-Id: I995576ab240c6f228716fa075d3513b47c41e7c1
2014-07-05 14:26:30 -07:00
Monty Taylor
0fb121893d Remove salt
We don't really use the salt infrastructure that we set up, which means
it's cruft. Go ahead and admit that we don't use it and remove it from
our systems.

Change-Id: Ic43695719cbad30aded16ac480deb3dfd9b2a110
2014-07-04 21:44:44 -07:00
Monty Taylor
034f37c32a Use ansible instead of direct ssh calls
Instead of a shell script looping over ssh calls, use a simple
ansible playbook. The benefit this gets is that we can then also
script ad-hoc admin tasks either via playbooks or on the command
line. We can also then get rid of the almost entirely unused
salt infrastructure.

Change-Id: I53112bd1f61d94c0521a32016c8a47c8cf9e50f7
2014-07-04 10:01:08 -07:00
Jenkins
3a63262fa9 Merge "Remove obsolete #openstack-climate channel" 2014-07-04 11:30:23 +00:00
Jenkins
1105933c8f Merge "Only run relevant puppet apply tests" 2014-07-03 22:45:19 +00:00
Jenkins
629d146fea Merge "Add in wsgi.conf for os-loganalyze" 2014-07-03 22:15:46 +00:00
James E. Blair
38d62b3030 Only run relevant puppet apply tests
Associate an OS with each node defined in site.pp so that the puppet
apply test will only test nodes apprapriate to the node the test
is running on.

Fix remaining errors related to this test, and make it vote.

Also, start running the centos6 version regularly
(which is still non-voting).

Change-Id: I26722dce15589c982af5de36f8ccf42666d29f2b
2014-07-03 15:11:23 -07:00
Joshua Hesketh
df4f93b891 Add in wsgi.conf for os-loganalyze
Configure to use the read only swift creds that pair up with the read
write creds used to push the files.

Change-Id: I53252b3ed0d596b3fe36caef179f253bde1739cb
2014-07-03 14:24:25 -07:00
Spencer Krum
b65a2d3afc Allow site.pp to manage ca and ca_sever in puppet.conf
This allows us to set ca = false and ca_server = <fqdn> on the
new puppet 3 master.

Change-Id: Iba189bdc4bfb22fd23052f2570f52133ea184126
2014-07-02 15:01:17 -07:00
Spencer Krum
6adda92be8 Add node def for puppet3 master
This change modifies install_puppet.sh to accept a --three option
setting it to install the latest puppet available. It also creates
a node definition for the puppetmaster.o.o node, the new 3 master,
and the master of the future. Changes were made to various classes
to allow the pinning to version 2.x to be turned off.

Change-Id: I805d6dc50b9de0d8a99cf818d22d06c2dea6090a
2014-07-02 13:25:14 -07:00
Sergey Lukjanov
28e8be0700 Remove obsolete #openstack-climate channel
Transition period for climate-blazar renaming ended.

Change-Id: Ie9fcca1666f4a6f96e13d420044abee07826fada
2014-07-01 12:24:59 +04:00
Jenkins
c74e6edbc6 Merge "Groups portal infra code refactor" 2014-06-24 00:34:41 +00:00
K Jonathan Harker
ad431b7700 Move variables in manifests/site.pp to top of file
Put the variables defined in manifests/site.pp at the top of the file so
that they are in a known location after running csplit in test.sh and we
can prepend them to the puppet-apply top files.

A better solution would be to move this data into hiera, but this is not
sensitive data, so a move to hiera should wait until we a have solution
for a public hiera data repo separate from the private hiera data.

Change-Id: I509a8266462dfdf53e1727938e4fb043241166b6
2014-06-17 17:05:56 -07:00
Monty Taylor
73d0474196 Add lifeless to statusbot list
He's in APAC and also knows all of the Launchpad folks, so
communicating status is helpful.

Change-Id: Ibb43981b10abc93c947f20c48315fedce5fb6608
2014-06-14 19:38:38 -07:00
Monty Taylor
ec428260cc Add jhesketh to the magic statusbot acl list
He's in core now, he should be able to set status.

Change-Id: I673e27ca52c2b77c108d5b78a48ab91725c7b6bd
2014-06-14 19:37:13 -07:00
Clark Boylan
ce03505892 Add elasticsearch07
Add elasticsearch07 node. Move the elasticsearch discover node to
elasticsearch02 instead of 01 as we are moving away from 01 as part of
the 07 addition.

Change-Id: I2aa857ec4984ae1fc2f8e27f437f8ecc61d24fbd
2014-06-13 11:19:34 -07:00
Clark Boylan
327f1690da Be specific about which ES nodes are puppetable
Our Elasticsearch nodes need cinder volumes attached to them prior to
being puppeted. Make the regex for the elasticsearch nodes in site.pp
match only existing nodes so that new nodes can be created and have
their volumes attached before proper puppeting.

Change-Id: If12c67174315b8480998148e1325de59ecdba372
2014-06-12 16:21:56 -07:00
Marton Kiss
a33ba74399 Groups portal infra code refactor
This patch removes the drush make site building function from
groups-dev instance, and now directly fetch release tarballs from
http://tarballs.openstack.org/groups repository. With an advanced
multi-slot deployment architecture it prevents the typical Drupal
WSOD issues that randomly caused site malfunction when a request
arrived during installation. It also simplifies the
deployment steps using the standard drush aliases and
drush-dsd extension and supports local configuration variables
in local_settings.php file.

Change-Id: I73976a60e080d15b6f513db79fee46bcf468e302
2014-06-11 23:12:12 +02:00
Clark Boylan
2c4cdcd317 Add four new logstash-worker nodes.
We have fallen behind on our ability to index all the things. Logstash
itself appears to be the current bottleneck. Add four more nodes to help
combat the backlog.

Change-Id: I9d5a7474a801c58c1e8933b264172e72f2d0bb86
2014-06-09 17:19:25 -07:00
Sergey Lukjanov
1678da69c5 Add #openstack-sahara to status bot
Change-Id: Id97e9c97fec5393b8d1b09a48ce61c871f633867
2014-06-02 18:25:48 +04:00
Jonathan Harker
4f25a69a89 Run the puppet apply test (requires sudo)
The test.sh script is not currently being run in any jobs, this change
removes the redundant validation code that's also in the puppet-syntax
job and creates a puppet-apply-test job that runs the test.sh script.

Running `puppet apply --noop` requires sudo, otherwise it will give
errors about refusing to run commands as other users.

Change-Id: Ie6b278d98390a8a5dd8bb24899c8c4083f5755c9
2014-05-28 17:02:16 -07:00
Clark Boylan
dd33ac2e10 Pass tenant_name to zuul config.
Turns out that while keystoneclient does not require a tenant_name be
set swiftclient does require it so the fix to not pass a tenant_name to
zuul did not work.

Alex_Gaynor pointed out the reason using the normal tenant_name doesn't
work with Rax object store is that the tenant_name for object store in
Rax is different than the normal tenant_name. Use that different and
correct tenant_name instead.

Change-Id: I13d6b8dfc190d35178e7fc979d7feb58e7e1f8dd
2014-05-23 16:00:50 -07:00
Sergey Lukjanov
9b26f18aaa Rename climate to blazar
Keep old channel #openstack-climate for some transition period.

Change-Id: Iea44afdbd69a43e0d6c349587ecb50318130b8b4
2014-05-23 21:54:07 +00:00
Jenkins
6d2af6e0cb Merge "Adding #openstack-containers IRC bot" 2014-05-23 10:12:42 +00:00
Joshua Hesketh
564f8ab237 Configure swift credentials for workers to push to
This change adds credentials as set up by fungi (2014-04-05T03:16:46)
http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2014-04-05.log

This allows zuul to send signed URL's for workers to push files to on
a per-job basis.

This change will require a zuul restart, not just a reload!

Note this is unreverts the revert in
c1b98eaff618b8c739cc02f089ef6fe6c02cbcc3 but is slightly edited hence
the lack of a proper revert commit. The reason for the revert has been
debugged and was due to rax identity service not providing a catalog
entry for swift when a tenant name is provided.

Change-Id: I04d3207002f7422b9851515ee88a74b19dd2f248
2014-05-20 13:36:36 -07:00
Adrian Otto
eaf54f1bb3 Adding #openstack-containers IRC bot
This is a dedicated IRC channel for discussion
about Operating System container support in
OpenStack. The members of the OpenStack
Containers Team will idle in this channel:

https://wiki.openstack.org/wiki/Teams/Containers

Change-Id: I9ec77969f926e8ced912ce194f7d6fae6a0e2e4d
2014-05-17 10:10:28 -07:00
Clark Boylan
c1b98eaff6 Revert "Configure swift credentials for workers to push to"
This reverts commit 176f15cd48ceea7e990fc61362875a23524f963a.

Puppet and zuul aren't ready for this. The swift client throws an
exception. We need to restart zuul to fix a more pressing issue hence
this revert. Will debug when able to.

Change-Id: Ie4487d84b7f1495c7ec6fd31e4b5b90d6699a31d
2014-05-05 18:49:15 -07:00
Jenkins
30d8b49457 Merge "Configure swift credentials for workers to push to" 2014-05-05 22:14:52 +00:00
Monty Taylor
0d831d14d3 Fix a very strange dyslexia
Change-Id: I20680349b32d0d0fef5a74cf9f0ac3ef7e57d682
2014-05-02 16:34:09 -07:00
Monty Taylor
ae8714d7f1 Migrate to using trove db for pastebin
The current drizzle-based pastebin has gotten a bit slow. Move to using
remote trove db.

Change-Id: I62fd164af26199fd9ed69770fac73c3e2f6a5344
2014-05-02 22:32:32 +00:00
Jenkins
c1345a4a8a Merge "Fixes to nodepool installation on jenkins-dev.o.o" 2014-05-01 02:04:23 +00:00
James E. Blair
22486c7353 Plumb mysql host through gerrit.pp
Change-Id: I9fceda90f9e9a06bc738e70f35595b7f05426ab0
2014-04-28 15:32:15 -07:00
Monty Taylor
47a361d596 Manage gerritbot ssh key with puppet
I'm not sure this really needs a longer commit message.

Change-Id: If0a7db1b4a988fd181753748d5fc9d26cf1bb906
2014-04-28 12:18:42 -07:00
Joshua Hesketh
176f15cd48 Configure swift credentials for workers to push to
This change adds credentials as set up by fungi (2014-04-05T03:16:46)
http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2014-04-05.log

This allows zuul to send signed URL's for workers to push files to on
a per-job basis.

This change will require a zuul restart, not just a reload!

Change-Id: Ib198112db0493456345fec83b32fabf606d835d2
2014-04-25 12:07:24 -07:00
James E. Blair
398e1175f4 Fixes to nodepool installation on jenkins-dev.o.o
The patch to add nodepool to jenkins-dev (https://review.openstack.org/#/c/57333)
did not work.

There were a few issues with it:
  1. jenkins-dev.pp was passing literal strings to the nodepool module, instead it
     should be passing in the variables.
  2. jenkins-dev.pp was calling ::nodepool but puppet seems to think that it wants
     ::openstack_project::nodepool due to puppet's scoping weirdness :(
  3. The script to build nodepool machines needed the jenkins_dev_ssh_key.

Fixes to above issues:
  1. This is trivial, just passed the variables thru instead of literal strings.
  2. The nodepool.pp module is renamed to nodepool_prod.pp to prevent the scoping problem.
  3. We use the dev jenkins ssh key with dev nodepool by allowing the nodepool module
     to pass arbitrary env settings through the defaults file.

Change-Id: Id91053212f088079ff1b0f06ebdce5c381f5cd19
2014-04-25 11:52:08 -07:00
James E. Blair
0c5b2fad15 Support configurable mysql host in gerrit
In further support of using a trove db, remove the gerrit::mysql
module from review-dev.  Plumb mysql_host throughout and remove
the no longer necessary mysql_root_password.  Configure review-dev
to use mysql_host from hiera, but configure review to use
'localhost'.

Change-Id: Id13cea66601f80c5b17d2a4069a14f7b458ea09d
2014-04-23 10:31:37 -07:00
Clark Boylan
30c2f7d462 Remove puppet dashboard.
Puppet board has replaced puppet dashboard. Remove reference to puppet
dashboard as it shouldn't be used.

Change-Id: I5eeee2984729ef5d1b883b4762347d19786e28ed
2014-04-17 14:48:19 -07:00
Jenkins
94405f8f2b Merge "Remove activity-dev server" 2014-04-17 19:30:25 +00:00
James E. Blair
2f3956198a Remove activity-dev server
It is not in use currently.

Change-Id: Iac53d2af79249448f14211a0f4a4972e46e10703
2014-04-17 10:40:52 -07:00
Monty Taylor
edaa31ebbd Add keys and script for puppet over ssh
In anticipation of driving puppet over ssh, we need keys on the hosts
and the scripts on the master. Don't turn them on yet, because we want
to be able to do some by-hand testing of the mechanism.

Change-Id: I2c353777e2f8fb5a2e733ce405ba40427ce901e5
2014-04-15 20:24:16 -07:00
Clark Boylan
943ed2315f Zuul use dedicated ssh key.
Zuul should use a dedicated ssh key rather than piggy backing off of the
jenkins ssh key. This change makes this distinction clear and removes
one zuul use of the jenkins ssh key that is not necessary.

Change-Id: I74c811a8bf94838b285791e158f4e468513eaa3e
2014-04-15 10:54:52 -07:00
Clark Boylan
7d1a297e4b Make jenkins proposal jobs use dedicated user.
Switch all jenkins proposal jobs to a dedicated user with dedicated
credentials. This is being done to be more flexible and secure when it
comes to managing the scripts that make proposals to gerrit.

Change-Id: I2dbdd530bf5b64c14207f645512a1eb319681166
2014-04-14 15:38:31 -07:00
Clark Boylan
929ebfd170 Use snakeoil certs on numbered jenkins masters.
Make it easier to deploy jenkins masters by using snakeoil certs on
numbered jenkins masters. This also simplifies the process of replacing
certs as make-ssl-cert can easily regenerate snakeoil certs for us.

Change-Id: I4966b1e502e0edf4f6fad25f06b9bacca25c5951
2014-04-09 14:50:19 -07:00
Marton Kiss
4ec69a09d8 Openstackid Mysql connection Hiera variables
Add two puppet hiera variables to override defaults for
openstackid-dev in site.pp to access remote user profile database
(mysql_user, db_name). Now we can use the following
mysql parameters for connection:
- openstackid_dev_ss_mysql_host: mysql host
- openstackid_dev_ss_mysql_password: password for mysql account
- openstackid_dev_ss_mysql_user: user name of mysql account
- openstackid_dev_ss_db_name: database name

Change-Id: Ied0e3918d117c13ef53763461242e5380f0bab00
2014-04-07 17:44:39 +02:00
Jenkins
32d3abef76 Merge "Revert "Enable Gearman as default on Jenkins slaves"" 2014-03-31 22:54:06 +00:00
James E. Blair
8af61da62f Revert "Enable Gearman as default on Jenkins slaves"
SERVER: Invalid relationship: File[/var/lib/jenkins/hudson.plugins.gearman.GearmanPluginConfig.xml] { require => Class[Zuul] }, because Class[Zuul]doesn't seem to be in the catalog

This reverts commit 2a9cd677e4eb6a60787ebc73bd8fd9e1b6f26b34
2014-03-31 22:49:41 +00:00
Jenkins
d302b1b698 Merge "Enable Gearman as default on Jenkins slaves" 2014-03-31 16:29:50 +00:00
Jenkins
b219665c28 Merge "Remove obsolete static job workers" 2014-03-30 22:27:43 +00:00