16 Commits

Author SHA1 Message Date
Clark Boylan
a57925c9ed Upgrade elasticsearch to 1.7.6
This can be done as a rolling upgrade as we are currently running 1.7.5
which shares a majory version with 1.7.6.

Change-Id: Icd759967dc8a6a09d4db0a78d2dfef72042c171e
2017-01-03 14:46:54 -08:00
Clark Boylan
6fcc8d3f26 Upgrade elasticsearch to 1.7.5
There have been a couple bug fix release between latest 1.7.x and 1.7.3
which we are running. Pull in this fixes in hopes that it makes shard
allocation more reliable.

Change-Id: I48f46663c73cb178ca3cf95a166b3df2ea121459
2016-05-04 12:06:58 -07:00
Jenkins
af13b1960e Merge "Switch from a cron to curator" 2015-11-16 21:30:50 +00:00
K Jonathan Harker
1d1b206620 Switch from a cron to curator
Instead of running curl in a cron, switch to the logstash::curator class
that runs a slightly more intelligent script with logging in /var/log.

Change-Id: Iba3c0726b789dcbcf58f2da071e6fa93d4cd67ae
Depends-On: Ic49c8649a0ad849a8d72900425e589e40b82a37f
2015-11-16 12:05:19 -08:00
Clark Boylan
285e90e16d Set indices.breaker.fielddata.limit to 70% on ES
We had to bump indices.breaker.fielddata.limit to 70% against the
running cluster in order for the elastic-recheck queries to run without
erroring due to not enough memory for the @timestamp field. Add this to
the config so that it persists across cluster installs.

Change-Id: Ia2f9c2ffff166bf3cc8f32c90b230249b3392406
Depends-On: I46c0cb5157aae40a0029ff1b425ecc663d171768
2015-11-13 12:08:14 -08:00
Clark Boylan
723a574ef8 Allow CORS against elasticsearch API
Since we do fine grained access control to the API via a proxy that
limits what requests can be made from anywhere go ahead and allow CORS
requests from anywhere as they won't be able to write data anyways.

This is useful so that you can test a new version of $tool hosted
locally against the actual cluster or for admins to run admin tools
hosted locally against the cluster.

Change-Id: I774d0ad0b246315794ab387acc39c41c7cfac3cd
Depends-On: I0aa8d5167c770c1024b7596da582d6cc089b1b47
2015-11-12 11:24:38 -08:00
Clark Boylan
11b297faf2 Update ES to 1.7.3
The actual upgrade is handled by ansible but this will enforce the
correct version for any new host builds. This should be merged after the
ansible managed upgrade is complete.

Change-Id: Ieda9cdffe7486d24f17012de5fc094d18a45ec78
2015-11-09 09:45:39 -08:00
Jenkins
2966b2e221 Merge "Move out server class from elasticsearch_node.pp" 2015-10-24 23:46:09 +00:00
Mikhail S Medvedev
053b4c657b Move out server class from elasticsearch_node.pp
Change-Id: Ic49e815e7346280de31891eb11faccb7cad3adad
Story: 2000172
Spec: http://specs.openstack.org/openstack-infra/infra-specs/specs/server_base_template_refactor.htm
2015-07-09 15:36:44 -05:00
K Jonathan Harker
78cde77666 Remove unused cron
Change-Id: Ife90f2210f3cdc1621e55251957a986a6d198941
2015-06-29 11:45:19 -07:00
Clark Boylan
411844b50a Shift elasticsearch retention to 10 days.
We were keeping 14 days of elasticsearch indexes. Unfortunately we are
running out of available disk space on our elasticsearch nodes and do
not have quota to add more volumes to increase available disk. Shift to
10 days of retention while we sort out how to get more disk.

Change-Id: I011d4f16f9a74e2eb7665ef66d2193e5fda476b7
2014-06-16 13:18:11 -07:00
Jeremy Stanley
cde1a4abee Create new 30g heap elasticsearch workers
Temporarily leave the old node names in place with an override to
16g heap size, but pass the openstack_project default of 30g into
the elasticsearch module as a new variable. Also don't actually
configure the new servers to install and run elasticsearch yet,
since there are manual steps needed to prep the additional block
devices on them.

Change-Id: I9fca2115c8996ae2f1a6c605bcdf16a8b6342a24
2014-02-25 00:04:37 +00:00
Clark Boylan
5a8351635d Upgrade to logstash 1.3.3 and ElasticSearch 0.90.9
Upgrade to logstash 1.3.3 and ElasticSearch 0.90.9. Logstash 1.3.3 comes
with many performance improvements and bug fixes and needs ElasticSearch
0.90.9 to use the ElasticSearch output so upgrade ES as well. These
upgrades come with config changes for logstash as well to use new
features and disable others that are not wanted.

Change-Id: Ia180c5edfa755214a801f2cad06f091125814b0e
2014-02-11 17:07:34 -08:00
Ryan Lane
712349aae7 Configure elasticsearch via a hash
Change-Id: I70067e371d930dcecab6e85399465b4c3bf67ab2
2013-10-21 18:48:29 -07:00
Clark Boylan
0423b59c6c Upgrade elasticsearch to version 0.90.3.
* modules/openstack_project/manifests/elasticsearch.pp: Upgrade
elasticsearch to 0.90.3 to support logstash 1.2.1. This version is
required to use the elasticsearch output in logstash 1.2.1.

* modules/elasticsearch/files/elasticsearch.mapping.json: Add setting
for new "message" field in logstash 1.2.1's new schema.

Change-Id: Ice1a8613ea11aeb954c600752a2de745dd56ef4b
2013-10-21 10:06:28 -07:00
Ryan Lane
2f4c686edc Rename openstack_project::elasticsearch
To avoid namespace issues, rename openstack_project::elasticsearch
to openstack_project::elasticsearch_node.

Change-Id: Icd215cd0c13f06d279a477d0885d33665ce125ae
2013-10-18 16:22:13 -07:00