openstack/openstack-ansible-ops
Code Issues Proposed changes
1480703942
openstack-ansible-ops/elk_metrics_7x/roles/elastic_ilm/defaults/main.yml

21 lines
616 B
YAML
Raw Normal View History

Improvements to ILM Elasticsearch 7 introduced Index Lifecycle Management, my aim is to eventually replace curator with this. This update allows for a default ILM policy to be configured for all beats. The default policy is quite basic: an index will be written to for 15 days or until the index reaches 30GB in size, then it will remain in the hot stage (no performance degradation) for another 15 days, after which it will be deleted. This still may lead to a situation where the disk space on log nodes could be filled. ILM policies can be configured for each beat by overriding the defaults that are in the role for each beat. ILM is set up for beats when undergoing an update (elk_package_state="latest"). During ILM set up elasticsearch creates an ILM ready template for the beat and uses the ILM policy we provision to nodes for the initial ILM policy. Subsequent ILM policy updates use ES APIs. New ILM policy files are not provisioned to nodes outside of using the beat upgrade flags, so the policy file on the node may fall out of step. Change-Id: I2c5c3abd4bb65075f2377227cbbfe31b68b0dc38
2019-10-03 15:13:46 +01:00
beat_version: "7.3.2"
default_ilm_rollover_max_size: "30G"
default_ilm_rollover_max_age: "15d"
default_ilm_delete_min_age: "30d"
default_ilm_policy:
policy:
phases:
hot:
actions:
rollover:
max_size: "{{ default_ilm_rollover_max_size }}"
max_age: "{{ default_ilm_rollover_max_age }}"
delete:
min_age: "{{ default_ilm_delete_min_age }}"
actions:
delete: {}
default_ilm_policy_filename: "default-ilm-policy.json"
default_ilm_policy_file_location: "/tmp"
Explicitly use Elasticsearch data node for ILM index update When installing/configuring beats, it may be the case that elastic-logstash nodes are not running Elasticsearch directly, and so can't update the ILM config accordingly. This commit explicitly delegates the ILM config update to an elasticsearch data node, in order that this will always be possible. As Elasticsearch and logstash are by default installed together, the default behaviour is unchanged. Change-Id: I6833585b9a02eddc3c9a1179e487cdeaefa30ac2
2020-02-05 16:44:53 +00:00
Subdivision of elastic-logstash group It is currently assumed that the elasticsearch data nodes and the logstash nodes are co-located within the same container during deployment of the ELK stack. This commit is intended to remove that assumption by requiring specification of separate elastic and logstash groups. This is particularly important where instances of elasticsearch and logstash are separated for resource reasons, a common setup for an elastic cluster. The default setup for the elastic nodes has been simplified, with all non-coordinating nodes being both data and ingest nodes unless otherwise specified. This was previously defined algorithmically, with the elastic node setup tightly coupled to the placement of the logstash instances. Change-Id: Id06eb78b52705aefea9cfe1247f53bac58badd52
2020-02-14 14:28:03 +00:00
elastic_beat_no_proxy: "{{ hostvars[groups['elastic'][0]]['ansible_host'] }}"
Copy Permalink