From aaeee5a976b6e2c2648d7628b894a6f8ae628dc9 Mon Sep 17 00:00:00 2001 From: "James E. Blair" <jeblair@openstack.org> Date: Mon, 10 Mar 2014 23:34:37 -0700 Subject: [PATCH] Document bad-image procedure for nodepool Change-Id: I4044e74e396f4f63cece95893ffdd9c9b6fdc74a --- doc/source/nodepool.rst | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/doc/source/nodepool.rst b/doc/source/nodepool.rst index dc0a2b4727..e4a6f83356 100644 --- a/doc/source/nodepool.rst +++ b/doc/source/nodepool.rst @@ -32,3 +32,40 @@ image with cached data for use by devstack. Nodepool spins up new instances and tears down old as tests are queued up and completed, always maintaining a consistant number of available instances for tests up to the set limits of the CI infrastructure. + +Bad Images +========== + +Since nodepool takes a while to build images, and generally only does +it once per day, occasionally the images it produces may have +significant behavior changes from the previous versions. For +instance, a provider's base image or operating system package may +update, or some of the scripts or system configuration that we apply +to the images may change. If this occurs, it is easy to revert to the +last good image. + +Nodepool periodically deletes old images, however, it never deletes +the current or next most recent image in the ``ready`` state for any +image-provider combination. So if you find that the +``devstack-precise`` images for a single or all providers are +problematic, you can run:: + + $ sudo nodepool image-list + + +--------+--------------------+------------------------+----------------------------------------------------------+------------+--------------------------------------+--------------------------------------+----------+-------------+ + | ID | Provider | Image | Hostname | Version | Image ID | Server ID | State | Age (hours) | + +--------+--------------------+------------------------+----------------------------------------------------------+------------+--------------------------------------+--------------------------------------+----------+-------------+ + | 168655 | hpcloud-az2 | devstack-precise | devstack-precise-1394417686.template.openstack.org | 1394417686 | 387612 | 4909797 | ready | 26.83 | + | 168696 | hpcloud-az2 | devstack-precise | devstack-precise-1394514268.template.openstack.org | 1394514268 | 388782 | 4930213 | ready | 0.75 | + +--------+--------------------+------------------------+----------------------------------------------------------+------------+--------------------------------------+--------------------------------------+----------+-------------+ + +Image 168655 is the previous image and 168696 is the current image +(they are both marked as ``ready`` and the current image is simply the +image with the shortest age. Delete the problematic image with:: + + $ sudo nodepool delete-image 168696 + +Then the previous image, 168655, will become the current image and +nodepool will use it when creating new nodes. When nodepool next +creates an image, it will still retain 168655 since it will still be +considered the next-most-recent image.