372 Commits

Author SHA1 Message Date
Vikram Hosakote
ec3baf0428 Install Elasticsearch and Kibana packages using yum/apt-get
Test results of this patch set are at
http://paste.openstack.org/show/490450/.

Change-Id: I2957cc70fcca34706ec50a630d785f0630c3f5c0
Closes-Bug: #1553203
2016-03-18 04:05:30 +00:00
Jenkins
9c235f6b91 Merge "Change libvirt volume name to libvirtd" 2016-03-18 02:06:19 +00:00
Jenkins
2701f28309 Merge "Increase the docker parition for rht bases in gate" 2016-03-17 17:02:59 +00:00
Jenkins
3c0641b36b Merge "MariaDB lights out recovery" 2016-03-17 06:24:32 +00:00
SamYaple
2aaaed770e MariaDB lights out recovery
This playbook only matters for multinode since AIO can recover from
power outage without additional configuration.

DocImpact
Implements: blueprint mariadb-lights-out
Change-Id: I903c3bcd069af39814bcabcef37684b1f043391f
2016-03-16 15:53:44 +00:00
Paul Bourke
e0688a762a Increase the docker parition for rht bases in gate
Currently the binary build for the oraclelinux base is just exceeding
the 10GB limit we have set here. Centos is at just under 9GB so will not
be long before it reaches this limit also.

According to infra -
(http://docs.openstack.org/infra/system-config/contribute-cloud.html)
gate VMs should have a 80GB disk, so 20GB for the docker partition does
not seem excessive here.

TrivialFix

Change-Id: I4d7fb240ea90e1e58f8f8046dd7acd0b9502c20d
2016-03-16 11:00:18 +00:00
Martin André
10b8cce268 Remove useless dependency on ifconfig
We can use the ip command to achieve the same thing.

TrivialFix

Change-Id: I903467c90495388cbc592b56d96bd3a4675df430
2016-03-16 09:23:30 +09:00
SamYaple
b98bc3a458 Change libvirt volume name to libvirtd
This change is a "futureproofing" thing. It has already been discussed
that libvirt should not be a child of nova and should be removed out
to the base docker directory (just like openvswitch isn't a child of
neutron). That is not going to happen this cycle but when it does we
can't change the name of the volume. This updates the volumes to the
proper name of libvirtd. This is in contrast with the libvirtlogd
volume that will be needed in newton due to libvirt 1.3

Of note, the container can remain named nova_libvirt since we can
change that on the fly later without breaking instances.

This wont break liberty as named_volumes are not backported yet.

TrivialFix

Change-Id: I16cf9e1b1dbba9b5a9f5cc883494580e276d4f72
2016-03-15 22:41:00 +00:00
SamYaple
8b3f0d0857 Use ubuntu Mitaka repo
The rabbitmq-server package is upgraded to 3.5.7 in cloud-archive
so we update centos to match

The xen-utils package now needs an explict version, xen-utils-4.6 is
what is provided by cloud-archive mitaka

Libvirt 1.3 is in the ubuntu cloud-archive. This has a new daemon for
logging that needs further implementation in newton. For now, it has
been disabled within the qemu.conf

Co-Authored-By: Jeffrey Zhang <jeffrey.zhang@99cloud.net>
TrivialFix
Change-Id: I84217824817c484b6800a64cbd7767b127a3098e
2016-03-14 15:52:48 +00:00
Éric Lemoine
32a4cadac9 Add cron image and playbook
Add cron image and playbook for logrotate.

The "common" Ansible playbook includes configuration files for
logrotate. At this point the operator cannot customize/override the
logrotate configuration.

Closes-Bug: #1553244
Change-Id: Ic9fdda9a273c9ccd90502f0acc7614d2c7157dca
2016-03-09 13:53:55 +01:00
Steven Dake
77a67deeb6 Rename validate-docker-execute with .sh extension
TrivialFix

Change-Id: I3ad67fd802f7fe3ed7deb88eddebe520ff410a4f
2016-03-08 22:14:09 -07:00
Jenkins
f14b931286 Merge "Copy the logs out of the container" 2016-03-09 04:17:50 +00:00
Jeffrey Zhang
12d6eeceb5 Copy the logs out of the container
Heka logs almost done. When testing, The collected logs can copy
to the /tmp/logs folder, which will be handle by the gate. This
will be very useful for our gate debug.

TrivialFix

Change-Id: I06ca8ebd13933bfd15cb2fb5e53fc3038b17b8c3
2016-03-09 09:34:59 +08:00
prithiv
caa3f90c83 Checks the installation of curl command, Nova, Neutron and OpenStack packages
related-bug: 1552433

Change-Id: If89e2a1bc959039fc7088c8295e88a199eabbedc
2016-03-08 20:35:50 +00:00
Jenkins
0af1df1eb7 Merge "Add documentation for two-VIP and TLS blueprints" 2016-03-08 16:48:14 +00:00
Dave McCowan
6d9cf739ca Add documentation for two-VIP and TLS blueprints
Relates-to: blueprint kolla-ssl

Change-Id: Id82d22c2195be5d897c781f28087a1ea422b8bea
2016-03-08 09:27:26 -05:00
Sam Yaple
99db1e9193 Add a new tool to help find the latest versions
As we will be switching to pinning versions in Kolla this tool will
help remove the 'manual' part of that equation by looking up the
versions we want.

The idea would be to eventually turn this into an autoproposal bot so
we only ever need to approve changes.

More work can be done here like automatically updating the
kolla-build.conf

Co-Authored-By: Martin André <m.andre@redhat.com>
Partially-Implements: blueprint upgrade-kolla

Change-Id: I052159850b2579ca6db79be71672be382be8a3a3
2016-03-08 11:45:47 +09:00
Jenkins
7e58ea219a Merge "Add two more examples of openrc for use with public endpoints" 2016-03-06 15:23:26 +00:00
Dave McCowan
25f99ad930 Add two more examples of openrc for use with public endpoints
Relates-to: blueprint kolla-ssl

Change-Id: I6d9531ce4513ed06f1d13081418b167ad264aaff
2016-03-05 20:45:10 -05:00
Jenkins
01b0be02b6 Merge "Use alphabetical order in cleanup-containers" 2016-03-05 15:36:50 +00:00
Jenkins
08101fd465 Merge "Make Heka send logs to Elasticsearch" 2016-03-05 15:36:44 +00:00
Éric Lemoine
97ae06341a Use alphabetical order in cleanup-containers
TrivialFix
Change-Id: I95f2e6d17ba7960b2f87344f30b7884d621f9ecb
2016-03-05 11:51:49 +01:00
Éric Lemoine
491aff0b88 Make Heka send logs to Elasticsearch
This patch includes changes relative to integrating Heka with
Elasticsearch and Kibana.

The main change is the addition of an Heka ElasticSearchOutput plugin
to make Heka send the logs it collects to Elasticsearch.

Since Logstash is not used the enable_elk deploy variable is renamed
to enable_central_logging.

If enable_central_logging is false then Elasticsearch and Kibana are
not started, and Heka won't attempt to send logs to Elasticsearch.

By default enable_central_logging is set to false. If
enable_central_logging is set to true after deployment then the Heka
container needs to be recreated (for Heka to get the new
configuration).

The Kibana configuration used property names that are deprecated in
Kibana 4.2. This is changed to use non-deprecated property names.

Previously logs read from files and from Syslog had a different Type
in Heka. This is changed to always use "log" for the Type. In this
way just one index instead of two is used in Elasticsearch, making
things easier to the user on the visualization side.

The HAProxy configuration is changed to add entries for Kibana.
Kibana server is now accessible via the internal VIP, and also via
the external VIP if there's one configured.

The HAProxy configuration is changed to add an entry for
Elasticsearch. So Elasticsearch is now accessible via the internal
VIP. Heka uses that channel for communicating with Elasticsearch.

Note that currently the Heka logs include "Plugin
elasticsearch_output" errors when Heka starts. This occurs when Heka
starts processing logs while Elasticsearch is not yet started. These
are transient errors that go away when Elasticsearch is ready. And
with buffering enabled on the ElasticSearchOuput plugin logs will be
buffered and then retransmitted when Elasticsearch is ready.

Change-Id: I6ff7a4f0ad04c4c666e174693a35ff49914280bb
Implements: blueprint central-logging-service
2016-03-05 11:51:49 +01:00
SamYaple
d5d1b1b3dd heatclient has invalid json validation picks up
We shouldn't be scanning .tox dir anyway.

TrivialFix

Change-Id: If570abdfa69375091751199dcce637682e8de1ef
2016-03-02 19:40:09 +00:00
Jenkins
76c28f9698 Merge "Update cleanup-containers to remove ceph containers" 2016-03-01 14:16:04 +00:00
Mauricio Lima
f1b8eec20d Update cleanup-containers to remove ceph containers
TrivialFix

Change-Id: Ia47429a38503d4982049c3fa1fc6fca4a8000fbd
2016-03-01 06:58:46 -05:00
Allen Gao
235e95885b Move ironic-discoverd to ironic-inspector
Closes-Bug: #1551052
Change-Id: Ic226287bbf1f1e8d5cc2e1d80ce2975448b1f95c
2016-02-29 11:47:52 +08:00
Jenkins
c81097df8b Merge "Change kolla_internal_address variable" 2016-02-27 17:50:52 +00:00
SamYaple
d3cfb2052a Change kolla_internal_address variable
Due to poor planning on our variable names we have a situation where
we have "internal_address" which must be a VIP, but "external_address"
which should be a DNS name. Now with two vips "external_vip_address"
is a new variable.

This corrects that issue by deprecating kolla_internal_address and
replacing it with 4 nicely named variables.

kolla_internal_vip_address
kolla_internal_fqdn
kolla_external_vip_address
kolla_external_fqdn

The default behaviour will remain the same, and the way the variable
inheritance is setup the kolla_internal_address variable can still be
set in globals.yml and propogate out to these 4 new variables like it
normally would, but all reference to kolla_internal_address has been
completely removed.

Change-Id: I4556dcdbf4d91a8d2751981ef9c64bad44a719e5
Partially-Implements: blueprint ssl-kolla
2016-02-26 20:00:09 +00:00
Dave McCowan
fd28087229 Add Ansible scripts to generate TLS certificates for testing
Working towards the blueprint that will add TLS protection
for the external endpoints, kolla needs certificates.

When kolla deploys OpenStack, the external VIP will need
a server side certifcate.  Clients that access those endpoints will
need the public CA certificate that signed that certificate.

This ansible script will create these two certificates to make
it easy to use TLS in a test environment.  The generated
certificate files are:

/etc/kolla/certificates/haproxy.pem  (server side certificate)
/etc/kolla/certificates/haproxy-ca.pem (CA certificate)

The generated certificates are not suitable for use in a
production environment, but will be useful for testing and
verifying operations.

Partially-implements: blueprint ssl-kolla

Change-Id: I208777f9e5eee3bfb06810c7b18a2727beda234d
2016-02-26 14:09:31 -05:00
Jeffrey Zhang
d26c259754 Add the default empty reconfigure.yml file to all roles
DocImpact

Partially-implements: bp kolla-reconfig
Change-Id: I9738e80960bcfbef18d1ef1b7942f81c45684e85
2016-02-25 09:32:53 +08:00
Éric Lemoine
0417844b8d Remove Rsyslog entirely
Partially implements: blueprint heka

Change-Id: I1322d2dc870e6f8fe052926995d993e8a08a25db
2016-02-23 01:45:23 -08:00
Jenkins
eb3e99399b Merge "Add elasticsearch repo to gate" 2016-02-23 01:22:54 +00:00
SamYaple
85600cd428 Add elasticsearch repo to gate
TrivialFix

Change-Id: I0f083654df901fb765e9517308ac94316859d971
2016-02-22 22:28:50 +00:00
Jenkins
c01ed04b2d Merge "Make cleanup-containers remove bootstrap_heka" 2016-02-22 18:00:36 +00:00
Éric Lemoine
b3ee41a288 Make cleanup-containers remove Heka volumes
TrivialFix
Partially implements: blueprint heka

Change-Id: Ie8f393515c464020f079db41c05d023319920136
2016-02-22 17:30:57 +01:00
Éric Lemoine
fadd8a3ded Make cleanup-containers remove bootstrap_heka
TrivialFix
Partially implements: blueprint heka

Change-Id: I5f91056590250f2a477018c1773905b412536f24
2016-02-22 13:56:29 +01:00
Éric Lemoine
916925c9a8 Add Heka to common role
Partially implements: blueprint heka
Change-Id: I9ddad09b3f16b323c7ed535243458b0de6620cdb
2016-02-19 20:32:00 +00:00
SamYaple
0cb1d969f4 Quick gate fix
I accidentally merged a patch with +W that does not work on all gates
that infra uses. Specifically it breaks on RAX for unknown reasons.

This is a quick patch to not fully revert, but allow the gate to stay
green while I work out the issue

Change-Id: I02fc66bbc53103f2d71c648d5e2959f10f0130c0
Partially-Implements: blueprint functional-testing-gate
2016-02-19 17:59:30 +00:00
Jenkins
e613cb816a Merge "Keep consistent with others for ironic containers" 2016-02-19 17:43:05 +00:00
Allen Gao
75e5782400 Keep consistent with others for ironic containers
Make the containers and volumes names of ironic to be
consistent with others.

Partially-implements: blueprint ironic-container
Change-Id: If78b020c81157a948c5c3283f1410bcb6e5c1e61
2016-02-19 20:30:59 +08:00
SamYaple
ae4407a2ab Launch VM in gate
Change-Id: I18b1015ca62a352e193ef01bd49cd6d6f9372193
Partially-Implements: blueprint functional-testing-gate
2016-02-18 19:55:24 +00:00
Jeffrey Zhang
29833132d8 Also remove the volumes when cleaning up containers
TrivialFix

Change-Id: Ia52451d2c0de09ee9dcf79211389a3b1ce4ae738
2016-02-18 09:06:44 +01:00
Jenkins
ff44f227c3 Merge "Adds support for cleaning up bootstrap containers in the toos/cleanup-containers script." 2016-02-18 01:14:38 +00:00
Jeffrey Zhang
d99de6b973 Fix the mistral services name typo
missing a "_"

TrivialFix

Change-Id: Id7624f4f35c3c53c068d7c6646b363332363c3d5
2016-02-17 19:21:25 +08:00
Shaun Smekel
6ac7839a21 Adds support for cleaning up bootstrap containers in the
toos/cleanup-containers script.

Dynamic Ceph OSD bootstrap containers also included.

Change-Id: Ifb5b54c5b78a894ed6e8b9980a6eb9919e52364c
Closes-Bug: #1515544
Signed-off-by: Shaun Smekel <shaun.smekel@theorem.net.au>
2016-02-17 18:49:55 +11:00
SamYaple
2c45bef065 Reduce gate resource usage
Change-Id: Ic6cd45ec708efe4f573e19f3bc910717ac3b4fbb
Partially-Implements: blueprint functional-testing-gate
2016-02-16 15:12:38 +00:00
Jenkins
dda86e6d03 Merge "Fix detect_distro" 2016-02-14 20:23:26 +00:00
Jenkins
a2c317f16d Merge "Fix non-root deploys" 2016-02-14 20:23:20 +00:00
Allen Gao
a5a0a830ef rabbitmq_data container is removed
TrivialFix

Change-Id: I5168191182c249df68fd0b5c70a778f223326974
2016-02-14 20:18:55 +08:00