opendev/system-config
Code Issues Proposed changes

Clean up logstash Jenkins console logs.

Browse Source 
* modules/openstack_project/files/logstash/log-pusher.py: Change
@message to event_messsage. Using @message here appears to confuse
logstash and results in duplicate events.

* modules/openstack_project/templates/logstash/agent.conf.erb: Change
@message to event_messsage. Using @message here appears to confuse
logstash and results in duplicate events. Also drop the opening and
closing pre tags that end up in the html log files.

Change-Id: I3cf679dc644392cfb8ef9002cf149d8b449ad4f5
Reviewed-on: https://review.openstack.org/26518
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
This commit is contained in:
Clark Boylan 2013-04-09 11:48:55 -07:00 committed by Jenkins
parent 1f95be17a3
commit 3ca843d707
2 changed files with 8 additions and 2 deletions
modules/openstack_project
files/logstash
log-pusher.py
templates/logstash
agent.conf.erb

2
modules/openstack_project/files/logstash/log-pusher.py

@ -92,7 +92,7 @@ class LogRetriever(threading.Thread):
for line in log_lines:
out_event = {}
out_event["@fields"] = fields
out_event["@message"] = line
out_event["event_message"] = line
self.logq.put(out_event)
def _parse_fields(self, event):

8
modules/openstack_project/templates/logstash/agent.conf.erb

@ -2,7 +2,7 @@ input {
pipe {
command => "python3 /usr/local/bin/log-pusher.py -r -z tcp://jenkins.openstack.org:8888 -l http://logs.openstack.org -f console.html"
format => "json"
message_format => "%{@message}"
message_format => "%{event_message}"
tags => ["jenkins", "console"]
type => "jenkins_console"
}
@ -10,6 +10,12 @@ input {
# You can check grok patterns at http://grokdebug.herokuapp.com/
filter {
grep {
type => "jenkins_console"
# Drop matches.
negate => true
match => ["@message", "^</?pre>$"]
}
grok {
type => "jenkins_console"
pattern => [ "%{DATESTAMP:logdate} \| %{GREEDYDATA:logmessage}" ]