Only enabled services logs should be added to cron container
and be inclued in cron.json.
This PS adds logic to only copy enabled services cron files.
Change-Id: Id6303a4c77a0230599e11be165489ba46dba65e0
Closes-bug: #1623712
Kolla's default heka pipline may not satisfy user's requirements,
for example, sending openstack log to outer log analyze system.
So it is necessary to add heka custom configuration for define
user's encoders, outputs and other plugins.
Change-Id: I48bd8d7e0afbc2d023c49c83041f87a04970bbb6
Closes-Bug: #1611164
This changed introduces 4 new parameters to be able to use an existing
elasticsearch service for central logging.
* elasticsearch_address - address of elasticsearch server
* elasticsearch_protocol - protocol (HTTP/HTTPS) used by elasticsearch server
* enable_elasticsearch - deploy elasticsearch container
* enable_kibana - deploy kibana container
Closes-bug: #1584861
Change-Id: Ia1ff9ae8b6d9929c3826da02693d1e2fc9ea2522
Previous work on Watcher added the Docker images, this
change adds the ansible configuration.
There is support for HA, via haproxy to balance across the
Watcher API hosts.
There is also a hook into nova.conf to conditionally add
Nova compute Host metrics via Ceilometer if Watcher is enabled.
This defaults to enabled false.
Change-Id: I8763528bb6ff12943b810212c71396d2d7cf6836
Partial-bug: #1598929
Partially-implements: bp watcher
Signed-off-by: Dave Walker (Daviey) <email@daviey.com>
If one is using an existing or alternative mysql deployment and sets
enable_mariadb to false, heka would fail to deploy.
TrivialFix
Change-Id: Ieb793e11a40358e5d4fc1f3955f200e2cddef4f5
Added general_log to ansible/roles/mariadb/templates/galera.cnf.j2
to improve mariadb logging.
This will be helpful to debug mariadb issues especially when
mariadb is scaled.
Test results of this patch set are at:
http://paste.openstack.org/show/492852/
Change-Id: I80438d1bbdd1ed2a1f47489c6f9c45b8107340a0
Closes-Bug: #1563668
Currently Heka writes the keepalived logs in
/var/log/kolla/haproxy/keepalived.log.
This commit changes this to /var/log/kolla/keepalived/keepalived.log.
Closes-Bug: #1565499
Change-Id: I3033097bd77ddbf72948697b34a6a499ea903083
Proposed patch to change keystone and heka log dir
from /var/log/kolla/apache2/ to /var/log/kolla/keystone/
Closes-Bug: #1560620
Change-Id: I70c65ceba5a301cc56880313ca86f01bd35676cb
This is consistent with the patch set in
https://review.openstack.org/#/c/296012/ for stable/mitaka
The new lua parser for libvirtd.log so that libvirtd.log can be
parsed in the Kibana dashboard will be implemented in a separate
patch set (in both master and backported to Mitaka).
TrivialFix
Related-Bug: 1560298
Related-Id: I1bc8bdb3180f30dc69af28a35fa057ac801bde0f
Change-Id: I59938a4e99abc676c391e535405417701aae2767
Swift and heka were broken by an optimization that resulted in
a regression in commit:
0a1486a2c73275009f7158c911e7bfd683e40a58
This patch fixes that problem.
Change-Id: I02c6cf4de4f807c3ec66e2d4f59f182d0b864ea3
Closes-Bug: #1560788
Changes location of Horizon log, they will be stored on common log volume
kolla_logs.
Change-Id: Ie9d56999a83efd05ab7c3dcb00b4dc42c9bce8f8
Closes-Bug: 1560250
These values are optional only when the services are not enabled.
If the file does not exist we should not warn, but rather inform.
Ceph-mon is an exception here since its bootstrap process means
the files may or may not exist initially.
TrivialFix
Change-Id: Ic02bece76d480e99deecf612036f37abb5604135
Now that we have reconfigure we dont have to copy over all config
files at the same time.
TrivialFix
Change-Id: I866be0f05e604151e081974a64f17bf65787f604
Add cron image and playbook for logrotate.
The "common" Ansible playbook includes configuration files for
logrotate. At this point the operator cannot customize/override the
logrotate configuration.
Closes-Bug: #1553244
Change-Id: Ic9fdda9a273c9ccd90502f0acc7614d2c7157dca
Heka logs almost done. When testing, The collected logs can copy
to the /tmp/logs folder, which will be handle by the gate. This
will be very useful for our gate debug.
TrivialFix
Change-Id: I06ca8ebd13933bfd15cb2fb5e53fc3038b17b8c3
This patch includes changes relative to integrating Heka with
Elasticsearch and Kibana.
The main change is the addition of an Heka ElasticSearchOutput plugin
to make Heka send the logs it collects to Elasticsearch.
Since Logstash is not used the enable_elk deploy variable is renamed
to enable_central_logging.
If enable_central_logging is false then Elasticsearch and Kibana are
not started, and Heka won't attempt to send logs to Elasticsearch.
By default enable_central_logging is set to false. If
enable_central_logging is set to true after deployment then the Heka
container needs to be recreated (for Heka to get the new
configuration).
The Kibana configuration used property names that are deprecated in
Kibana 4.2. This is changed to use non-deprecated property names.
Previously logs read from files and from Syslog had a different Type
in Heka. This is changed to always use "log" for the Type. In this
way just one index instead of two is used in Elasticsearch, making
things easier to the user on the visualization side.
The HAProxy configuration is changed to add entries for Kibana.
Kibana server is now accessible via the internal VIP, and also via
the external VIP if there's one configured.
The HAProxy configuration is changed to add an entry for
Elasticsearch. So Elasticsearch is now accessible via the internal
VIP. Heka uses that channel for communicating with Elasticsearch.
Note that currently the Heka logs include "Plugin
elasticsearch_output" errors when Heka starts. This occurs when Heka
starts processing logs while Elasticsearch is not yet started. These
are transient errors that go away when Elasticsearch is ready. And
with buffering enabled on the ElasticSearchOuput plugin logs will be
buffered and then retransmitted when Elasticsearch is ready.
Change-Id: I6ff7a4f0ad04c4c666e174693a35ff49914280bb
Implements: blueprint central-logging-service
Due to poor planning on our variable names we have a situation where
we have "internal_address" which must be a VIP, but "external_address"
which should be a DNS name. Now with two vips "external_vip_address"
is a new variable.
This corrects that issue by deprecating kolla_internal_address and
replacing it with 4 nicely named variables.
kolla_internal_vip_address
kolla_internal_fqdn
kolla_external_vip_address
kolla_external_fqdn
The default behaviour will remain the same, and the way the variable
inheritance is setup the kolla_internal_address variable can still be
set in globals.yml and propogate out to these 4 new variables like it
normally would, but all reference to kolla_internal_address has been
completely removed.
Change-Id: I4556dcdbf4d91a8d2751981ef9c64bad44a719e5
Partially-Implements: blueprint ssl-kolla
Currently Heka fails to parse the RabbitMQ logs. There are two
problems:
1. The rabbit-sasl.log file is ignored but the file_match expression
does not match.
2. The delimiter used in the RegexSplitter makes Heka stop on the
very first log entry. '\n\n(=[^=]+====' (with two \n's) is
a better delimiter. deliver_incomplete_final is used to get the
final log entry.
TrivialFix
Change-Id: I94720340d5b2d6fd5d7641b9ff58733f6cd882ee
HAProxy: change to use option forwardfor to pass origin IP address
to backend via X-Forwarded-For header
Keystone: Apache does the audit logs for keystone. Change the
LogFormat to display the passed address instead of the connection
address which is that of the load balancer.
Nova, Cinder, Glance: these services can make use of the address
passed in X-Forwarded-For. With this setting the API logs for
these services include the client IP address.
Change-Id: Ia861ecc11a7c7d463d0366586926d1a842853f69
Closes-Bug: #1548935
Swift uses Syslog, but it uses a custom log format. So this commit
adds a specific Heka decoder for Swift.
It also increases the log level from "warning" to "info" to make
Swift more verbose. Note that "info" is the default log level in
Swift.
And it disables the Heka configuration for Swift when "enable_swift"
is set to "no". This prevents Heka from creating 15 empty Swift log
files in the logs volume.
Partially implements: blueprint heka
Change-Id: If7a7d0707e71be2957178e2d45b5de51b788232e
