59 Commits

Author SHA1 Message Date
SamYaple
1f371e2717 Allow per node configuration with augments
This type of per node configuration is required to support things like
availability zones for nova. As always, if this file doesnt exist it
doesnt get used so this change is safe.

TrivialFix

Change-Id: Iff8172af522c2c96e5f2c173b24a5dfd4d522ed2
2016-03-17 17:18:00 +00:00
SamYaple
ff91d53c76 Refactor glance config tasks
This now matches the way we copy confis in other services.

TrivialFix

Change-Id: Ia9c6d5f9a633c0b20b4d9a697863bd2753d39443
2016-03-17 17:18:00 +00:00
SamYaple
57124620ab Fix Keystone v3 and Horizon
After our switch to keystone-manage bootstrap Horizon is not happy
due to v3 not being setup correctly. This patch fixes that

This also includes removal of unused variables (transforms them into
endpoint url variables)

TrivialFix
Change-Id: I1e04db8c24049f80e974c063f03068a2ab32a563
2016-03-03 15:44:53 +00:00
Jeffrey Zhang
75ce1e447d Reconfigure glance service
Change-Id: Ic9e98941711a5fd1b8546de58d278a0b00cf350b
Partially-implements: bp kolla-reconfig
2016-03-02 23:50:21 +00:00
SamYaple
d3cfb2052a Change kolla_internal_address variable
Due to poor planning on our variable names we have a situation where
we have "internal_address" which must be a VIP, but "external_address"
which should be a DNS name. Now with two vips "external_vip_address"
is a new variable.

This corrects that issue by deprecating kolla_internal_address and
replacing it with 4 nicely named variables.

kolla_internal_vip_address
kolla_internal_fqdn
kolla_external_vip_address
kolla_external_fqdn

The default behaviour will remain the same, and the way the variable
inheritance is setup the kolla_internal_address variable can still be
set in globals.yml and propogate out to these 4 new variables like it
normally would, but all reference to kolla_internal_address has been
completely removed.

Change-Id: I4556dcdbf4d91a8d2751981ef9c64bad44a719e5
Partially-Implements: blueprint ssl-kolla
2016-02-26 20:00:09 +00:00
Jeffrey Zhang
d26c259754 Add the default empty reconfigure.yml file to all roles
DocImpact

Partially-implements: bp kolla-reconfig
Change-Id: I9738e80960bcfbef18d1ef1b7942f81c45684e85
2016-02-25 09:32:53 +08:00
Jeff Peeler
301602d8f1 Playbook for glance upgrade
New playbook for glance service upgrade.

Change-Id: I759e4eddf669112f752fe07d6b99a4bb9593d97f
Implements: blueprint upgrade-glance
Partially-Implements: blueprint upgrade-kolla
2016-02-22 10:25:51 -05:00
Éric Lemoine
5de471f61e Make Heka collect Glance logs
Partially implements: blueprint heka
Change-Id: I9e5ba649bc45356822138ad223430f4b93722448
2016-02-19 21:49:21 +00:00
Jenkins
6e622fa534 Merge "Move socket binding to named_volume" 2016-02-17 02:06:26 +00:00
SamYaple
690e6853de Move socket binding to named_volume
The extend_start.sh script for rsyslog is removed as it is no longer
needed. Docker no longer binds to /dev/log or /run/kolla/log

Closes-Bug: #1544545
Change-Id: Ic0a323a26ee4e9e15baf4598285844a8a4955f23
2016-02-16 14:42:41 +00:00
Dave McCowan
1cedf77f19 Use variables to specify http or https when constructing URLs
To allow for TLS to protect the service endpoints, the protocol
in the URLs for the endpoints will be either http or https.

This patch removes the hardcoded values of http and replaces them
with variables that can be adjusted accordingly in future patches.

Change-Id: Ibca6f8aac09c65115d1ac9957410e7f81ac7671e
Partially-implements: blueprint ssl-kolla
2016-02-15 09:48:58 -05:00
SamYaple
e8d66766f0 Fix gate for docker 1.10
Docker 1.10 has broken the gate and this patch will correct that
breakage.

The issue comes with rsyslog. Due to a commit in Docker 1.10 [1] we
must change the way we get the log socket for rsyslog. The /dev/
folder will no longer populate as we used it. So instead we simply
make a new socket in a path we control and share that to the correct
location in the containers.

Additionally, adjust the gate for new Docker daemon.

[1] https://github.com/docker/docker/pull/16639
Partially-Implements: blueprint kolla-upgrade
Change-Id: I881a2ecdf6d7b35991e1d38a3f3e60d022d6577f
2016-02-06 06:48:53 +00:00
SamYaple
80b7266ed1 Rename kolla_ansible to kolla_toolbox
This change is needed for clarity. We have a kolla-ansible script.
We have a kolla-mesos repo. We plan to have a kolla-ansible repo.
Already we have had far too much confusion about whether we are
talking about the container or the project. Naming this kolla-toolbox
eliminates all of that confusion and its probably a bit more accurate
of a name too.

Closes-Bug: #1541053
Change-Id: I8fd1f49d5a22b36ede5b10f46b9fe02ddda9007e
2016-02-02 18:12:15 +00:00
SamYaple
6d8959a962 Update all bootstraps to use Label
Add bootstrap label to all bootstrap containers to ensure that when
the a new container is launched a difference is seen between it and
the bootstrap container since we cannot rely on ENV variables for
this. This only affects mariadb at this stage, but it is needed to
ensure rabbitmq works when we switch to named volumes.

Change-Id: Ia022af26212d2e5445c06149848831037a508407
Closes-Bug: #1538136
2016-01-28 20:26:41 +00:00
Michal Jastrzebski
375965ddfb Add stub upgrade.yml
After introduction of pull action and turing every main.yml into
{{action}}.yml we lost ability to perform upgrade

Change-Id: Ie9fa2cd083b061033abc733fba53d54f9c55e393
Fixes-Bug: #1538210
2016-01-26 11:52:19 -06:00
SamYaple
e0fc4cb3bd Fix missing mountpoint in glance bootstrap
Change-Id: I637eef710c0c4964a440e21e3ac15f8fca4b1830
Closes-Bug: #1519965
2016-01-06 22:43:44 +00:00
SamYaple
b60bb946d0 Allow glance role to pull images
Change-Id: Ib5c22c7416bdc7ce83123b79fa7b08afda7595bc
Partially-Implements: blueprint pre-pull-images
2016-01-06 04:23:28 +00:00
SamYaple
8af400733d Convert to named volumes instead of data container
This intentionally leaves out rabbitmq from this patchset. It will
require additional work to remove its data container

UpgradeImpact
Partially-Implements: blueprint docker-named-volumes

Change-Id: Id68b8e43a3c077ef4f4f4d67ea34d0692e66eef7
2016-01-03 16:44:44 +00:00
SamYaple
9e5e51dead Convert Glance to kolla_docker
Change-Id: I1372bdb14818b5a8920e8b0d1483fd481fb83589
Partially-Implements: blueprint kolla-docker-module
2015-12-31 16:38:56 +00:00
SamYaple
ed82afa8e9 Simplify config creation
Convert config creation from a playbook to an action_plugin. This
reduces the complexity and confusion while retaining the same augment
structure and flexibility.

This allows us to remove the 0-byte files as requirements. They will
still be used if they are present (this means we require additional
documentation around them).

DocImpact
Closes-Bug: #1528430
Change-Id: I2c789f6be9f195c7771ca093a6d59499564b4740
2015-12-22 04:28:53 +00:00
Michal Jastrzebski
aab1bd36c8 Sanity check for glance
This runs first sanity check for glance. After glance is deployed
it checks images.list(). Also consist few fixes for previous patches

Change-Id: I03d05d246302d8411b2e94c94ca7fe046c00d735
Partially-Implements: blueprint sanity-check-container
2015-12-02 14:19:01 -06:00
Jenkins
359f8e83d6 Merge "Remove the two different auth profiles" 2015-11-26 11:51:05 +00:00
Sam Yaple
970617b80a Make the database json variable more readable
Additionally remove tty from the container cleanup docker section. It
was added in a sed in a previous patchset by serves no purpose.

Change-Id: Ib617870616bca687f72ffaa44b2e9a3a11ef1011
Partially-Implements: blueprint cleanup-playbooks
2015-11-23 17:24:22 +00:00
Hui Kang
790bf336d5 Fix bootstrap for services
The bootstrap tasks could be started on any node listed in the
site.yml

The issue is that all the tasks must run on the same node, and the
only node that all the tasks can run on is the 'api' node due to
needing to start the bootstrap container which binds in the configs.

delegate_to is required to ensure that the proper node gets the task.

Co-Authored-By: Sam Yaple <sam@yaple.net>
Change-Id: Ica04ab366777a571a92212ac22c482f1881d4ecd
Closes-Bug: #1513598
Related-Bug: #1513439
2015-11-23 17:09:36 +00:00
Sam Yaple
8c1e7061f8 Remove the two different auth profiles
Ceph can function just fine generating the keys before the pools have
been created so we can apply the proper permissions to the auth string
ahead of time. This allows us to not require additional steps to add a
cache tier on the fly in the future.

Change-Id: I8214c567fb7c337f95d908c5699d1da922bfa1a6
Closes-Bug: #1518475
2015-11-20 23:24:53 +00:00
Jenkins
96667c9241 Merge "Move data container to start.yml" 2015-11-20 15:01:52 +00:00
Paul Bourke
988c4f23f1 Fix issues in Glance filesystem backend
The drop root change for Glance highlighted the fact that we were
binding volumes from glance_data into the wrong container - it was
glance_registry whereas it should be glance_api. This would result in
all images being lost if the glance_api container happens to restart.

Also, we need a sudoers file to chown the file backend dir to the glance
user.

Change-Id: If04337045bb94b3126e48d1f5bf0ea29e20373ae
Closes-Bug: #1516729
2015-11-17 09:42:39 +00:00
Sam Yaple
564f8f2bd4 Move data container to start.yml
The original reason for having the data container in the bootstrap.yml
was to keep consistency with rabbitmq and mariadb, but in light of
this bug we need to move the data container out of the bootstrap.yml
role since we will be limiting the hosts that those tasks can run on.

Change-Id: I7f2f5979b01807275908699d1243756cb97d3588
Partial-Bug: #1513598
2015-11-09 10:47:16 +00:00
Sam Yaple
6b28343017 Incorrect parsed variable name
database_user_create was not correctly referenced when parsing the
variable names. This could never actually lead to a situation that
reported a false change, but it could break an operation if you were
using the --step option with ansible and skipped the database create
task.

TrivialFix
Backport: Liberty

Change-Id: Idf69fffcc3814f509448ccea11b7d175f074ccf1
2015-11-09 10:47:15 +00:00
Jenkins
037bdca42d Merge "bootstrap takes place on the wrong node" 2015-11-06 03:32:36 +00:00
Sam Yaple
e2bc9dcf79 bootstrap takes place on the wrong node
The bootstrap must occur on the nova-api node due to binding in the
nova-api directory (same goes for all other services)

Closes-Bug: #1513439
Backport: Liberty
Change-Id: Iab88b49712828085e4d7e7f85e6d8f0b7999a9bf
2015-11-05 16:30:44 +00:00
Sam Yaple
347730cec8 Remove vip for rabbitmq
Adjust all the configs to list all the rabbitmq hosts rather than
running rabbitmq through the VIP. This is made possible by clusterer
which has already merged.

Change-Id: I5db48f5f10ec68f4c8863a29bc13984f6845a4f9
Partially-Implements: blueprint rabbitmq-clusterer
2015-11-05 15:43:20 +00:00
Sam Yaple
b27880aa77 Automate setting up a cache tier
Introduces a new flag to bootstrap cache devices

DocImpact
Partially-Implements: blueprint ceph-improvements

Change-Id: I09b5a0d5c61b3465237e5f01dc10120725561cd3
2015-10-22 10:51:44 +00:00
Sam Yaple
1d60be3770 Allow ceph pool names to be configurable
Change-Id: Ic408ef7aec7f1c95396a0ccf0209b104908a3f21
Partially-Implements: blueprint ceph-improvements
2015-10-22 10:51:44 +00:00
Sam Yaple
6932ca48b8 Allow specifying erasure as the pool type
Change-Id: I581efd7c0d138cb78ab3afd755521f3fb375c0ee
Partially-Implements: blueprint ceph-improvements
2015-10-22 10:51:44 +00:00
Sam Yaple
b2ac6e80cd Add tty to containers
sudo requires a tty to function by default on centos. Instead of
tweaking the sudo conf we can just add a tty. This has the added
advantage of making the containers more friendly if you have to
`docker exec -i <container> bash` into them.

Change-Id: If97a02ca1d37c243a787d98ade54bde8d641aecd
backport: liberty
Partially-Implements: blueprint functional-testing-gate
2015-10-14 10:54:33 +00:00
Sam Yaple
c5be228be8 Bump ansible to 1.9.2
Ansible 1.9.2 contains the fix needed for docker-py >1.2.0

This is needed for some gate reasons, but it is also a good version
bump because it fixes a few issues with delegate_to.

Change-Id: Iafbabb3b0232620849d0548c5cd9d8d316c2b0f3
backport: liberty
Partially-Implements: blueprint functional-testing-gate
2015-10-13 15:13:34 +00:00
Sam Yaple
94162bd08f Bring Kolla inline with FHS
This brings Kolla images inline with FHS and should make finding
locations of things more consistent and reliable with the linux world
at large.

Change-Id: Iece5b4da4bace0fb8b1f41a65ab2c852ec73e6f8
Closes-Bug: #1485742
2015-10-06 03:30:53 +00:00
Sam Yaple
5223965b61 Fix ceph after config-external removal
Change-Id: I46dc2f02dda8219878eee3755b0891f4c3450682
Partially-Implements: blueprint replace-config-external
2015-09-29 08:39:43 +00:00
Sam Yaple
99c7eb1997 Make OpenStack use Ceph
This implements all the openstack pieces needed to make ceph work.

DocImpact
Change-Id: I1d24476a966602cf955e5ef872b0efb01319894a
Partially-Implements: blueprint ceph-container
Implements: blueprint kolla-live-migration
2015-09-25 20:11:18 +00:00
Michal Jastrzebski (inc0)
943ffd76ed Glance logging to rsyslog
Configure glance to use rsyslog container as logging mechanism

Change-Id: I60f604d7a85bb4309b26be03cd6b8aac14840fe7
Implements: bp logging-container
2015-09-24 12:45:46 +02:00
rthallisey
3fa0aa2ddc Replace config-external with a JSON file for Glance
Change-Id: I14187db31a009c150424ea28b854a78c9ae3941d
Partially-Implements: blueprint replace-config-external
2015-09-18 15:19:06 -04:00
Sam Yaple
f42528acaf Properly check failed status
No need for a new task.

Change-Id: I958f87ac9a1434a2f6052e0271f37e5f3109775e
Closes-Bug: #1492337
2015-09-06 11:21:44 +00:00
Vladislav Belogrudov
7f81dbc85b Check exit status of bootstrap containers
Currently bootstrap containers are waited to exit but are not
checked for exit status and ansible runs further tasks. If
bootstrapping fails we notice it at much later time.

Change-Id: I137fc11b0f9d1f03d2ded08a213e8dbd62741f92
Closes-Bug: #1492337
2015-09-04 19:47:54 +03:00
Paul Bourke
f5c996e77d Fix typo in task bootstrap task names
Not quite halloween yet...

Change-Id: Ib1973eea0b2dddc0e0845731d1212c4d4447cca5
Closes-Bug: #1490502
2015-08-31 11:27:10 +01:00
Steven Dake
98b98924ee Use database_user_create in the register operation
The register operation was registering user_database which is a global
variable in group_vars/all.yml.  This results in glance not being deployable
because user_database is overwrriten at keystone task registration time.

The result of this is the playbooks fail to deploy.

Change-Id: Ic69725fb09aa0a368ec5b0d8f47bfc768c29c70c
Closes-Bug: #1486876
2015-08-20 00:49:31 -07:00
Steven Dake
bd5e5bbb64 Use one data container for all data volumes
The previous method of using the API image for a data container
worked, but resulted in more data being bindmounted and copied
into the system.  This patch resolves that by essentially using
a distro-type as a FROM in a Dockerfile to generate
distro-type-data.  Further all of the data containers used
throughout ansible are modified to use this new data container.

Change-Id: I8846573d4f2d4b98d4f46c770bfefc6d4c5cd0b5
Partially-Implements: blueprint one-data-container
2015-08-18 20:52:27 -07:00
Sam Yaple
a6f3a1073a Fix volumes_from for the ansible playbooks
Change-Id: Ib4b563d20d8e5ea5b81fb89e9ee4943487d7e6cd
Closes-Bug: #1485297
2015-08-16 09:10:58 +00:00
Ryan Hallisey
59b016f490 Glance-registry is missing a messaging config
Change-Id: Ibf20d74f2ea5583bc1c183d14329e37661c20721
Closes-Bug: #1483793
2015-08-12 07:20:06 -04:00
Vladislav Belogrudov
305192152d Glance bootstrap container mounts wrong config folder
Glance bootstrap container is based on glance-api. Ansible playbook
incorrectly mounts
    "{{ node_config_directory }}/glance" to "/opt/kolla/glance"
where it should mount
    "{{ node_config_directory }}/glance-api" to "/opt/kolla/glance-api".
After start bootstrap container fails to find configuration and exits.

Change-Id: Id2e5ddb2191cfee7db57dd0aec2f06fc617aa9d6
Closes-Bug: #1483757
2015-08-11 17:07:01 +03:00