The journalbeat configuration has been updated to make it
similar to all other beats. This change updates our config
so that it is functional with the latest journalbeat release.
Change-Id: Ic70a031bdeb57f2f5439763a3bf9f6b7001e6a31
Signed-off-by: Kevin Carter <kevin@cloudnull.com>
When installing ES and Logstash the system version of java needs to
match the expected version of java ES and Logstash will use. This
change, uses the `update-alternatives` command to set the java version
to the expected value when more than one java exists on a system.
The nessisity for this change came from OS level upgrades within
environments running OLD versions of ES. Upon upgrading the base OS
our playbooks could not complete an upgrade of ES which was due to the
java expectations. Once the alternantives were set accordingly the
upgrade completed without issues.
Change-Id: I9025967f723ee17940e11789f503e342cdad6f2a
Signed-off-by: cloudnull <kevin@cloudnull.com>
Appears we ran into this bug on 16.04:
https://bugs.launchpad.net/ubuntu/+source/debian-installer/+bug/1816876
and the fix is in xenial-updates. Switching all the ubuntu installer
kernels to the updated ones so that we can make sure we're using
latest and greatest.
Change-Id: I68cccbb142c8b4c657a60929ad21e8164d07a1ef
monitorstack for openstack services was added to site however to this is
not quite ready yet so its been removed from site and will remain as a
beta capability for the time being.
Change-Id: Ia9a8af49c288bf04c02b3be6b4179c9a2fb07076
Signed-off-by: cloudnull <kevin@cloudnull.com>
The service name for packetbeat needs to be set correctly, this does
that.
Change-Id: I39c10914ba2d0f16b6ebb94da480ad13f455a08f
Signed-off-by: cloudnull <kevin@cloudnull.com>
This change allows the larger os metrics installation process to be
selective. A dynamic group will be created and hosts will be added to it
when the host is within a known target group and is using systemd
(required for now).
Change-Id: I225ec5b5ffe4aa8ba403624f9ebe8c9eebed9fee
Signed-off-by: cloudnull <kevin@cloudnull.com>
This change will ensure that the upper and lower memory limits are set
to whole numbers when dynamically setting the memory limits. Before this
change a memory limit could be a float, and in certain versions of
ansible + python the option could evaluate to 0, which would break
everything.
Change-Id: Ibfb8af60db6566937cbf77243b0e6848d542665d
Signed-off-by: cloudnull <kevin@cloudnull.com>
The grafana role will now deploy and setup the grafana datasources using
the API as expected. API users will also be created for admin, viewer, editor.
The es config for grafana has been udpated to correct issues where the system
expected a publically accessible lb to handle grafana traffic back to an es
cluster. When the grafana role deploys the traefik lb will now be used within
the grafana deployment to ensure grafana is able to deploy against an es cluster.
Change-Id: Iae3a5c2ab1b98390110d37f33b074156d32bb684
Signed-off-by: Kevin Carter <kevin@cloudnull.com>
The monitorstack data collection can export data into elasticsearch.
A playbook has been added to deploy the data collection probes which
will leverage systemd-timers to run the probes on regular intervals. The
systemd timers will be deployed per-probe and run within the utility,
compute, and memcached hosts. Any place the probes are deployed an
isolated user will ensure to fence the probes from the cluster and limit
access. OpenStack probes will only be deployed when an openstack-sdk
clouds config is found within the system.
Change-Id: Ic5cd5fd51a7e0763c0a2db40af4150b8851bc748
Signed-off-by: cloudnull <kevin@cloudnull.com>
In the event that both NGINX and apache are co-existing on the same machine
the status port check for both platforms will be the same, and that will
cause one of the services to not start. This change increments the NGINX
check port to ensure there are no conflicts.
Change-Id: I03d5d351fff2d6926f35ca860c01f5a075de42aa
Signed-off-by: Kevin Carter <kevin@cloudnull.com>
This change ensures we are not creating a port conflict for apache/nginx when
status is enabled. This change also ensures dashboards are created correctly
resolving an issue with index-patterns containing a regex.
Change-Id: I8228fc9832d02518d2db843c96abf6dffc63bdfc
Signed-off-by: Kevin Carter <kevin@cloudnull.com>
This change removes the {h,q}_mem options in favor of a new variable which
clearly states the upper and lower limits for a given deployment. This change
also makes these options a lot more conservative by default which will allow
the deployment to better run on shared infra.
Change-Id: I169f457198c11edc4881a04df65312f6c4f67feb
Signed-off-by: Kevin Carter <kevin@cloudnull.com>
This change creates a new option to enable or disbale rollup jobs. This
is also providing the default basic index patterns for kibana index
patterns and elastic indexes.
Change-Id: I60e96a2cdbe27de760b54c4d9d43bcde4d09bbf5
Signed-off-by: cloudnull <kevin@cloudnull.com>
This change adds the metering sub group and points all of our systems at
that instead of assuming they can all be built on "hosts" and
"log_hosts".
Change-Id: I3cb951456f82090fad1b02cda470055691c0ae35
Signed-off-by: cloudnull <kevin@cloudnull.com>
The beat service starts were overly complex this simplifies them.
Change-Id: I3d059d06c5e7839b6ce2e0d39de3e88299a2e6bb
Signed-off-by: cloudnull <kevin@cloudnull.com>
This change will allow logstash to ingest metrics from collectd. New
options have been added to enable the deployment and configure it.
Change-Id: I995c0db69fc68d5f5bcae27ce16956876368e2a8
Signed-off-by: cloudnull <kevin@cloudnull.com>
This commit adds the ability for users to enable Designate (DNSaaS)
in a MNAIO build. This service is disabled by default.
Change-Id: I36e10922b6fe8e5cba3cc929e2b91b59507c210d
This change makes it possible for users to set the `elastic_heap_size_default`
value. Before this change, the option was unreachable due to a series of facts
ganerated template values. The options `elastic_heap_size` or `logstash_heap_size`
have also been exposed giving deployers the ability to define service specific
heap sizes as needed.
Change-Id: Ida3a57fdcff388f8e4bb3f325b787205a6183970
Signed-off-by: Kevin Carter <kevin@cloudnull.com>
This patch adds support for deploying the ML2/Open Virtual Network (OVN)
plugin for Neutron to an MNAIO deployment. A new var,
osa_enable_networking_ovn, can be set to lay down the appropriate bits.
Change-Id: Ib5bd4e0c20be62ddbf0bff13c91d1918907bf230
With the introduction of the "infrustructure" panel and "canvas" becoming
stable, there's not a lot of reason to import the general beat dashboards.
The default dashboard are almost always in a state of disrepare and take a
long time to import on high traffic clusters.
This channge removes the default dashboard from the beat setup role by
default. If a deployer wishes to renable the default dashboards, or add any
other beat flags, the variable `elastic_setup_flags` can be used to extend
the setup.
Change-Id: If44845f53e4d0cb1e91ec804060316fb852b4bfa
Signed-off-by: Kevin Carter <kevin@cloudnull.com>
Before adding IP addresses to the CNF validate the entry is an IP
address using the `ipaddr` filter.
Change-Id: I9151b8118b92991b394c0fa7d81d407439f0f3c1
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This change updates the galera deployment options for kolide which will
allow it to co-exist on the same host as an elasticsearch environment.
Because the galera cluster for kolide is on the backend, and not user
facing the installation and setup of a VIP for the backend galera
cluster is not needed.
Change-Id: Ia5f581ebb028bf35c00dc7606b0eb1a12ad54670
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The queue path within logstash may be a symlink which will fail to mount
as tempfs. To ensure queue path can be tempfs, a readlink command is
used to fetch the true path, which will be used in a mount when nessisary.
Change-Id: I5fe6bf311e0621c98766ae458371b5f11f89a61f
Signed-off-by: Kevin Carter <kevin@cloudnull.com>
The option `skydive_service_setup_host` allows a user to define a
setup host target which could, or could not, be in the provided
inventory. Additionally a setup target host could also be simply
an IP reference. This change ensures that the playbooks and roles
respect the different setup host delegation node types by creating
in memory host entries and gathering facts on the dynamic
information when the target is not in inventory, is not in the
skydive_all group, or simply an IP.
Change-Id: I532abd7171ba9077759640e4bf18b9b517264426
Signed-off-by: Kevin Carter <kevin@cloudnull.com>
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This change implements namespaced variables and conditionals in needed
services. This will ensure systems running these playbooks are able to
be deployed in isolation without making osa specific assumptions.
Change-Id: Ia20b8514144f0b0bf925d405f06ef2ddc28f1003
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This change implements TLS configurations for skydive. All servers will
get their own certificates signed by the CA as setup on the
`skydive_service_setup_host` node.
> Documentation for TLS configuration can be found here: http://skydive.network/documentation/
Change-Id: I890b4a5d9076b0474ffe5649f9361cb6018c19fe
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This patch allows a deployer to override the service setup host
and python interpreter path so that the keystone service setup
can be done on a different host, typically the first utility
container in an openstack-ansible deployment.
Overrides can be provided in an integration inventory and an
example would be:
skydive_service_setup_host: "{{ groups['utility_all'][0] }}"
skydive_service_setup_host_python_interpreter: "/openstack/venvs/utility-{{ openstack_release }}/bin/python"
Change-Id: Id4faf605fd1f0de1262ed48166a8801b7e24a528
This config line prevents the downloaded binary version
of traefik from starting up, which in turn prevents the skydive
binary from starting up correctly.
Change-Id: Ic07a3c6170ff4e0ad60dbdb860eff0f330b677a1
Skydive fails to compile unless WITH_OPENCONTRAIL=false. This
is a known issue on ubuntu hosts from the upstream project.
Change-Id: Ie6d165eb128fbfa1cece7f031b8a2416339641d5
The readme was updated for correctness.
The make command when building skydive was changed from "install"
to "static" this was done to assist folks building skydive which
"should" allow it to be redistributable.
Change-Id: I01f8f677f1ccf54d8dd29ac8a2eb60c0206319a0
Signed-off-by: Kevin Carter <kevin@cloudnull.com>
This change will enable neutron network awareness and set the
auth-backend to to use keystone when `skydive_openstack_enabled` is
set to true. Options have been added to allow users to customize the
deployment, all of which have been documented in the default/main.yml
file.
Change-Id: Iab958c0631c7c396d218d8fb6302db03e5d9a5a6
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Signed-off-by: Kevin Carter <kevin@cloudnull.com>
The use of an overlay inventory has now been added to a stand alone
directory which should help deployers understand how this could work
within their environments. Inventory configuration has been updated to
show this working in the various projects that have tests and could
benefit from an overlay inventory.
Change-Id: I1681fce160b43e9127851f06ae05ff5eaabe852f
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Given the ops tools now have a skydive deployment capability there needs
curator needs to be able to detect the addition of skydive indexes and
build a curator policy accordingly.
This change adds the new retention policy to the overlay inventory
providing a sane default for most environments.
The retention action files have been updated to remove the "-" as an
index separator. This was done because not all indexes use a dash as a
divider.
Change-Id: I5b61720f27da00e0c3b92341355b09ea6c01caba
