Previously puppet would restart nodepool whenever the nodepool log
config changes. This is problematic because nodepool will try to do a
graceful shutdown which can take hours. During this time period no new
nodes are built. Stop doing this and rely on manual nodepool restarts
(which is status quo for any other nodepool changes).
Change-Id: I522070df96d4e455a2c478bcf832ae6bb8729bf2
The openstack_project::nodepool class does not expose vhost_name
parameter that is available in nodepool::init.pp class.
Change-Id: Ie975b4798d5219a3debde453e297bed3be72e9e6
Closes-Bug: 1383657
Individual nodepool .log files have now grown to being > 250mb. This
is a bit much to download and go through when you're trying to debug
an issue with a particular image.
8-hour rotations seems a bit more useful than 24-hour ... and perhaps
even that is a bit long. I don't think we need to extended the number
of rotations, I'm not sure really old logs are that helpful.
Change-Id: Ia788db809afe470bb9241f17411e4fa97d09ea11
In Ie0b269835ebb8effbac0285b782d8add7b47db32 I didn't consider how
difficult it is to get puppet to put in AllowOverride directives to
allow .htaccess to work. The puppetlabs-apache version we use does
not support "override" as an argument, so you can't easily add this to
allow apache to read htaccess. Upgrading is a big task because we are
so far behind (see I6fa5f3275a84ec4480169de562d1d4a656111814).
So deploy a full template for nodepool vhost that includes the config
options we need.
Additionally, the existing mimetype isn't set for the log-files
---
$ curl -I http://nodepool.openstack.org/image.log.2014-10-14
HTTP/1.1 200 OK
Date: Fri, 17 Oct 2014 00:02:39 GMT
Server: Apache/2.2.22 (Ubuntu)
Last-Modified: Tue, 14 Oct 2014 23:59:56 GMT
ETag: "f89bb-d458315-5056acfe33700"
Accept-Ranges: bytes
Content-Length: 222659349
---
I'm pretty sure this means it defaults to text/plain, but this makes
it explicit. This should also matches on the timestampped log files
per the rules of multiple extensions [1]
[1] http://httpd.apache.org/docs/2.2/mod/mod_mime.html#multipleext
Change-Id: I7fa2603f4160b06af71a515e655d4a705fa0c768
Add a .htaccess to the nodepool log directory to enable gzip
compression for text/text mime-types; e.g. the quite large .log files
that nodepool outputs there
Change-Id: Ie0b269835ebb8effbac0285b782d8add7b47db32
Keep nodepool's version of diskimage-builder up-to-date by installing
the latest version for pip. DIB releases at least weekly so we can get
update quickly via this install method.
Change-Id: I7e97388d890d37f1f86b860c2309ab1d63f76e41
When we install dib via pip the executables are put in /usr/local/bin
which was not in nodepool's PATH. The PATH is configured by the init
script so update the variable to include /usr/local/bin.
Change-Id: Ie456ecf667d330b925d3168a456d4ac84d1c48f4
Disk image builder requires root permissions to chroot and mount images.
Update the puppet manifest for nodepool to optionally enable
passwordless sudo access for the nodepool user. This defaults to being
allowed but can be toggled if this is deemed an unnecessary security
risk.
Change-Id: If0bf5f182d88c848cd2a64c5c75cc64cc0b42c58
For nodepool to use diskimage-builder, it needs a set of pre-reqs
on the nodepool host. Install them.
Also create /opt/nodepool_dib for building images. This is
deliberately placed on the non-ephemeral disk as new host builds have
a large partion mounted here. Testing shows test images were coming
in at about 2.6GB each. If we figure two images for precise and trusty
and one each for centos6 and f20 that is about 15GB.
Change-Id: I725cad999f795a556fd89818578b78b9114fb110
When attempting to install zuul & nodepool on a single machine
(typical for third party environments) you get the following error:
Duplicate declaration: Package[python-lxml] is already declared in file
/root/config/modules/zuul/manifests/init.pp at line 64;
cannot redeclare at /root/config/modules/nodepool/manifests/init.pp:39 on node ci.novalocal
Update the puppet scripts so that python-lxml do not conflict with each other.
Change-Id: I63453e46702d37fa845504f31fa006b0b9fc5330
Nodepool now depends indirectly on lxml via keystoneclient. To pip
install lxml you need libxml and libxslt development packages on the
system to provide headers so that the library bindings can be
compiled, so it's easier to just use the python-lxml system packages
instead.
Co-Authored-By: Jeremy Stanley <fungi@yuggoth.org>
Change-Id: Id5c2070250cd9474c83edaa6ed1119a5325ea913
Installing nodepool standalone has this error:
Could not find dependency Class[Pip] for Exec[install_nodepool] at
/root/config/modules/nodepool/manifests/init.pp:90
Add the missing include pip statement.
Change-Id: Id30814eb1788e7b7eb533794a26c231000e9b41c
The patch to add nodepool to jenkins-dev (https://review.openstack.org/#/c/57333)
did not work.
There were a few issues with it:
1. jenkins-dev.pp was passing literal strings to the nodepool module, instead it
should be passing in the variables.
2. jenkins-dev.pp was calling ::nodepool but puppet seems to think that it wants
::openstack_project::nodepool due to puppet's scoping weirdness :(
3. The script to build nodepool machines needed the jenkins_dev_ssh_key.
Fixes to above issues:
1. This is trivial, just passed the variables thru instead of literal strings.
2. The nodepool.pp module is renamed to nodepool_prod.pp to prevent the scoping problem.
3. We use the dev jenkins ssh key with dev nodepool by allowing the nodepool module
to pass arbitrary env settings through the defaults file.
Change-Id: Id91053212f088079ff1b0f06ebdce5c381f5cd19
Nodepool logs the creation of nodepool images to an image.log which
are only accessible to root users. This change updates the nodepool
puppet configuration to install apache and allows users to access
these image logs.
Change-Id: I867030c258d00ce017c69812c133f3419215d045
Nodepool is having its file descriptor limit bumped to 8192. Bump the
mysql connection limit to match to avoid having trouble on the other
side.
Change-Id: Ic164bddc860f8f569a2a0acb5290aaeb775bdd9a
Nodepool is running into file descriptor limits like Zuul. Again
compromise and bump the limit in the init script to give us breathing
room until the problem can be properly fixed.
Change-Id: Idd1fd679bdad1d3ea4951be7f9330214d7065027
The pip vs. easy_install interactions continually get the systems
into a weird state. pip install $path_to_repo, on the other hand,
works like a charm.
Left out two graphite installs because they are doing path manipulation
and I want to do further testing before touching that.
Change-Id: I373b29aca03f1ebd20e533ebaaf5de9ef2db017e
Puppetlabs-mysql 0.6.1 correctly removes the local ::1 root user in the
account_security manifest. Upgrade to this version to take advantage of
that. Do not upgrade to latest version (0.9.0) to minimize delta that
needs testing.
Change-Id: Ic8265733f1159f34ae0afcccdea4c7d8cd44e3cb
The version of puppetlabs-mysql that we use does not remove the local
::1 root user from the mysqld. Explicitly remove this user.
Change-Id: I626fcc77c75a29d3f3cab57217b714e68a30b468
Nodepool can end up with one connection per thread when dealing
with launches and deletions. If we have a lot of node turnover,
that's a lot of threads.
Change-Id: Ice483f5f9f3a47963357f9ff59bf4d735a18aa43
An upcoming change to nodepool supports logging the output from
the image creation process. Use it.
Change-Id: I3a45a30f813d0f7b9fff4f602b945d72f2dcda3e
