* Implements G1 GC optionally. The variable `elastic_g1gc_enabled` has
been added with a default of false. If this option is set true and the
system has more than 4GiB of RAM G1GC will be enabled.
* Adds new thread options
* Better constraints coordination nodes
* Interface recover speed has been limited
* Buffer size is now set correctly
* Serialize elk deployment so that upgrades are non-impacting
Change-Id: I89224eeaf4ed29c3bb1d7f8010b69503dbc74e11
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
To ensure we're not creating too many logs files the logrotate config
has been added to truncate and compress log files every other day
with a max retention priod of 5 days.
Change-Id: I0dd85a334dba48e9fce3aad67ead867e2f46cb02
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Systemd overrides have been added to the service unit files for all
beats and services. All of the playbooks have been updated to make them
look and feel uniform.
This also sets handlers within the playbooks so that we're improving the
idempotence.
Change-Id: I2dd3183dae4bfddc607cc74f9dfb7af115b80abc
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The following options will reduce cluster pressure and generally
improve search performance.
Change-Id: I1619680db1fd595503f0845b182d6f6ce4c59f3c
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The following change will ensure that the elastic-static is logging to
the journal and that systemd is able to report how well the elastic
slice is running.
Change-Id: I79a9074b5f14a41dec421d6691fd04c0e6be15b7
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The journalbeat shipper will now have a "journald" tag attached to it
which will then be used by logstash to identify items and add tags to
them based on the systemd slice.
To accomodate this load the logstash config has been updated to better
handle blowback and scale according to the capability of the underlying
hardware.
The logstash grok files have been moved into the files directory which
was done to speed up the time it takes to ship files to a given host.
Originally the task used to copy these files was using the template
module however none of these files have anything templated within them
so there's no need to run them through the template engine.
Change-Id: I4f0e50ac491b595c0d276f6e7292d2c6e61baa22
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This change will allow a deployer to directly ship data from
logstash into Kafka.
Change-Id: I5de0caf270c8ced8111ac099cb91a70814f80259
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The templates which generate the configuration for the various
components require access to the facts defining the cluster layout.
Running the setup*.yml playbooks with --tags config would not include
common_task_data_node_hosts.yml and templating would fail due to
undefined vars.
This change applys an 'always' tag to all uses of the include.
Change-Id: I6c55a60aef54f9bb26eb9a2af08315265daa82d9
This change breaks out the basic install and setup playbooks into
different playbooks. This is done to ensure an operator can easily rerun
any part of the playbook as they upgrade, change, or modify a setup over
the lifetime of the deployment.
Change-Id: I66c53fcb21880c950ea3fee202e7d2224dfdff3a
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The tasks with external deps should all have a retry so to cater for
cases of intermittent service interruptions. This change adds retires
and tags to of the tasks with external deps.
Change-Id: Icd0a8143347bcbffc2a1e0a02da92091b59ed483
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
To ensure users can upgrade packages the variable
`"{{ elk_package_state | default('present') }}"` has been added
to all package installs.
Change-Id: I0238d9e1ed991cb1480bd924f2d5a09687890da3
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Elasticsearch can be used as a smart load balancer for all traffic
which will remove the requirement for a VIP and move the cluster to a
mesh topology. All of the Kibana nodes will now run elasticsearch as
cordonator.
* Kibana will now connect to elasticsearch on localhost.
* All of the beats have been setup to use use the new mesh topology.
* jvm memory management has been updated to reflect the additional
services.
More on node assigments can be found here:
* https://www.elastic.co/guide/en/elasticsearch/reference/6.2/modules-node.html#modules-node
* The readme has been updated to reflect these changes.
Change-Id: I769e0251072f5dbde56fcce7753236d37d5c3b19
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The 30-elasticsearch-output.conf.j2 were renamed to
99-elasticsearch-output.conf.j2 in the templates directory but not in the
task "Drop Logstash conf for beats output" in installLogstash.yml playbook.
In addition to that, templates/extras file were created and also a task to create
/opt/logstash/patterns were added in the playbook.
Change-Id: I5ddd2694bf03726bf76ba30d34515b4b02f64234
This adds the ability to include logstash log parsing filters for
various openstack and service logs. These filters are disabled by
default and can be enabled by toggling the deploy_logstash_filters
variable.
Change-Id: I5c46f78f232d3fb604283ae623cd3975a8346c7c
When running a multi-node ELK stack (5+ nodes) the elasticsearch server
needs to be able to mark and identify nodes that will be data and master
nodes. This change ensures that the first set of available master nodes
is mark accordingly as both data and master and that all other nodes are
marked as cluster members with alternating nodes being marked as data
nodes. With this change the cluster will be able to grow as well as
handle large data sets more efficiently.
This change ensures that sharding is correctly started when the cluster
is deployed and that the beat services are using the recommended
sharding practices.
Change-Id: Id555132199f268b21aaa939a84760c744238dd2b
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This change allows EL to have customizable heap sizes which will
automatically set themselves to the recommended size as advised by the
elastic documentation.
https://www.elastic.co/guide/en/elasticsearch/reference/6.2/heap-size.html
Change-Id: I55d014dab37bf5e5a4806bd19d4e07e072f9fa7e
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
* Added options for the rollback plan so that if a rollback is executed
all beat packages will be removed.
* additional updates to streamline elk and fix container bindmounts,
the use of group information for metric and heartbeat information.
* Readme information has been fixed
Change-Id: Icd070259db5b19d289d10033b1f055125f56e18c
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This change ensures java is installed where logstash is installed and
changes the logstash data-directory to a bind mount which will ensure
we're not running into space issues inside of our logstash container.
Change-Id: I43f7fb4a8641150c05dcde88a94b65e7c7d2f83a
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Most of the changes in this PR are for style and to adapt the playbooks
so that the system can operate on a multi-node cloud.
Functional change includes the removal of mainline Java 8 in favor of
OpenJDK 8.
A site playbook was add to allow an operator to just run everything.
Old tools that no longer function within the stack have been removed.
Packetbeat was added to the install list
Auditbeat was added to the install list
All of the config files have been updated for the recent ElasticStack
6.x changes.
Change-Id: I01200ad4772ff200b9c5c93f8f121145dfb88170
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This addition is an updated of the curent elk_metrics which will install Elasticsearc, Logstash and Kibana 6.x.
It also include configuration guide for haproxy endpoints
Change-Id: Iac4dec6d17bc75433e5fe672f3b9781536b8e619
