Add firehose docs for ansible events
This commit adds documentation for the ansible events we're now publishing into the firehose. This includes full schema documentation explaining the topics and payloads. Change-Id: I85e27f1c27895c8c25bb34072481f4f75911d815
This commit is contained in:
Matthew Treinish
parent
1ef6088bb6
commit
3abe3e830d
@ -76,6 +76,8 @@ As of right now the following services publish messages to the firehose:
|
||||
+----------------+-----------------+---------------------------+
|
||||
| Service | Base Topic | Source of Messages |
|
||||
+================+=================+===========================+
|
||||
| ansible | ansible | `ansible_mqtt_plugin`_ |
|
||||
+----------------+-----------------+---------------------------+
|
||||
| gerrit | gerrit | `germqtt`_ |
|
||||
+----------------+-----------------+---------------------------+
|
||||
| launchpad | launchpad | `lpmqtt`_ |
|
||||
@ -86,6 +88,7 @@ As of right now the following services publish messages to the firehose:
|
||||
.. _germqtt: http://git.openstack.org/cgit/openstack-infra/germqtt/
|
||||
.. _lpmqtt: http://git.openstack.org/cgit/openstack-infra/lpmqtt/
|
||||
.. _subunit-gearman-worker: http://git.openstack.org/cgit/openstack-infra/puppet-subunit2sql/tree/files/subunit-gearman-worker.py
|
||||
.. _ansible_mqtt_plugin: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/files/puppetmaster/mqtt.py
|
||||
|
||||
For a full schema description see :ref:`firehose_schema`
|
||||
|
||||
|
||||
@ -149,3 +149,249 @@ An example is::
|
||||
'build_uuid': '45f7c1ddbfd74c6aba94662623bd61b8'
|
||||
'source_url': 'A url',
|
||||
}
|
||||
|
||||
Ansible
|
||||
=======
|
||||
|
||||
We have mqtt events emitted from ansible being run on
|
||||
:ref:`puppetmaster.openstack.org`. These events are generated using a
|
||||
`MQTT Ansible Callback Plugin`_.
|
||||
|
||||
.. _MQTT Ansible Callback Plugin: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/files/puppetmaster/mqtt.py
|
||||
|
||||
Topics
|
||||
------
|
||||
|
||||
The topics for ansible are a bit more involved than some of the other services
|
||||
publishing to firehose. It depends on the type of event that ansible just
|
||||
finished. There are 3 categories of events which have slightly different topic
|
||||
formulas (and payloads).
|
||||
|
||||
Playbook Events
|
||||
'''''''''''''''
|
||||
Whenever a playbook action occurs the callback plugin will emit an event for
|
||||
it. The topics for playbook events fall into this pattern::
|
||||
|
||||
ansible/playbook/<playbook uuid>/action/<playbook action>/<status>
|
||||
|
||||
``playbook uuid`` is pretty self explanatory here, it's the uuid ansible uses
|
||||
to uniquely identify the playbook being run. ``playbook action`` is the action
|
||||
that the event is for, this is either going to be ``start`` or ``finish``.
|
||||
``status`` is only set on ``finish`` and will be one of the following:
|
||||
|
||||
* ``OK``
|
||||
* ``FAILED``
|
||||
|
||||
to indicate whether the playbook succesfully executed or not.
|
||||
|
||||
Playbook Stats Events
|
||||
'''''''''''''''''''''
|
||||
|
||||
At the end of a playbook these events are emitted for each host that tasks were
|
||||
run on. The topics for these events fall into the following pattern::
|
||||
|
||||
ansible/playbook/<playbook uuid>/stats/<hostname>
|
||||
|
||||
In this case ``playbook uuid`` is the same as above and the internal ansible
|
||||
unique playbook identifier. ``hostname`` here is the host that ansible was
|
||||
running tasks on as part of the playbook.
|
||||
|
||||
Task Events
|
||||
'''''''''''
|
||||
|
||||
At the end of each individual task the callback plugin will emit an event. Those
|
||||
events' topics fall into the following pattern::
|
||||
|
||||
ansible/playbook/<playbook uuid>/task/<hostname>/<status>
|
||||
|
||||
``playbook uuid`` is the same as in the previous 2 event types. ``hostname`` is
|
||||
the hostname the task was executed on. ``status`` is the result of the task
|
||||
and will be one of the following:
|
||||
|
||||
* ``OK``
|
||||
* ``FAILED``
|
||||
* ``UNREACHABLE``
|
||||
|
||||
Payload
|
||||
-------
|
||||
|
||||
Just as with the topics the message payloads depend on the event type. Each
|
||||
event uses a JSON payload with slightly different fields.
|
||||
|
||||
Playbook Events
|
||||
'''''''''''''''
|
||||
|
||||
For playbook events the payload falls into this schema on playbook starts::
|
||||
|
||||
{
|
||||
"status": "OK",
|
||||
"host": <hostname>
|
||||
"session": <session id>,
|
||||
"playbook_name": <playbook name>,
|
||||
"playbook_id": <playbook uuid>,
|
||||
"ansible_type": "start",
|
||||
}
|
||||
|
||||
When a playbook finishes the payload is slightly smaller and the schema is::
|
||||
|
||||
{
|
||||
"playbook_id": <playbook uuid>,
|
||||
"playbook_name": <playbook name>,
|
||||
"status": <status>,
|
||||
}
|
||||
|
||||
In both cases ``playbook uuid`` is the same field from the topic.
|
||||
``playbook name`` is the human readable name for the playbook. If one is
|
||||
set in the playbook this will be that. ``status`` will be whether the
|
||||
playbook was successfully executed or not. It will always be ``OK`` on starts
|
||||
(otherwise the event isn't emitted) but on failures, just like in the topic,
|
||||
this will be one of the following:
|
||||
|
||||
* ``OK``
|
||||
* ``FAILED``
|
||||
|
||||
``session id`` is a UUID generated by the callback plugin to uniquely identify
|
||||
the execution of the playbook. ``hostname`` is the hostname where the ansible
|
||||
playbook was launched. (which is not necessarily where tasks are being run)
|
||||
|
||||
|
||||
An example of this from the system is for a start event::
|
||||
|
||||
{
|
||||
"status": "OK",
|
||||
"playbook_name": "localhost:!disabled",
|
||||
"ansible_type": "start",
|
||||
"host": "puppetmaster.openstack.org",
|
||||
"session": "14d6e568-2c75-11e7-bd24-bc764e048db9",
|
||||
"playbook_id": "5a95e9da-8d33-4dbb-a8b3-a77affc065d0"
|
||||
}
|
||||
|
||||
and for a finish::
|
||||
|
||||
{
|
||||
"status": "FAILED",
|
||||
"playbook_name": "compute*.ic.openstack.org:!disabled",
|
||||
"playbook_id": "b259ac6d-6cb5-4403-bb8d-0ff2131c3d7a"
|
||||
}
|
||||
|
||||
|
||||
Playbook Stats Events
|
||||
'''''''''''''''''''''
|
||||
|
||||
The schema for stats events is::
|
||||
|
||||
{
|
||||
"host": <hostname>,
|
||||
"ansible_host": <execute hostname>,
|
||||
"playbook_id": <playbook uuid>,
|
||||
"playbook_name": <playbook name>,
|
||||
"stats": {
|
||||
"unreachable": int,
|
||||
"skipped": int,
|
||||
"ok": int,
|
||||
"changed": int,
|
||||
"failures": int,
|
||||
}
|
||||
}
|
||||
|
||||
``playbook uuid`` is the same field from the topic. ``playbook name`` is the
|
||||
human readable name for the playbook. If one is set in the playbook this will be
|
||||
that. ``execute hostname`` is the hostname where the tasks were being executed,
|
||||
while ``hostname`` is the hostname where ansible launched the playbook. The
|
||||
``stats`` subdict contains the task status counts where the key is the tasks
|
||||
statuses.
|
||||
|
||||
An example from the running system is::
|
||||
|
||||
{
|
||||
"playbook_name": "compute*.ic.openstack.org:!disabled",
|
||||
"host": "puppetmaster.openstack.org",
|
||||
"stats": {
|
||||
"unreachable": 0,
|
||||
"skipped": 5,
|
||||
"ok": 13,
|
||||
"changed": 1,
|
||||
"failures": 0
|
||||
},
|
||||
"playbook_id": "b259ac6d-6cb5-4403-bb8d-0ff2131c3d7a",
|
||||
"ansible_host": "controller00.vanilla.ic.openstack.org"
|
||||
}
|
||||
|
||||
|
||||
Task Events
|
||||
'''''''''''
|
||||
|
||||
The schema for tasks events is::
|
||||
|
||||
{
|
||||
"status": <status>,
|
||||
"host": <hostname>,
|
||||
"ansible_host": <execute hostname>,
|
||||
"session": <session id>,
|
||||
"ansible_type": "task",
|
||||
"playbook_name": <playbook name>,
|
||||
"playbook_id": <playbook uuid>,
|
||||
"ansible_task": <task name>,
|
||||
"ansible_result": <ansible result>
|
||||
}
|
||||
|
||||
``playbook uuid`` is the same field from the topic. ``playbook name`` is the
|
||||
human readable name for the playbook. If one is set in the playbook this will be
|
||||
that. ``execute hostname`` is the hostname where the tasks were being executed,
|
||||
while ``hostname`` is the hostname where ansible launched the playbook. ``task
|
||||
name``, like the name implies, is the human readable name of the task executed.
|
||||
If one was specified in the playbook that will be the value. ``status`` is the
|
||||
result of the task and will be one of the following:
|
||||
|
||||
* ``OK``
|
||||
* ``FAILED``
|
||||
* ``UNREACHABLE``
|
||||
|
||||
``session id`` is a UUID generated by the callback plugin to uniquely identify
|
||||
the execution of the playbook.
|
||||
|
||||
``ansible result`` is a free form subdict that comes directly from ansible to
|
||||
completely describe the task that just finished. The structure here is fully
|
||||
dependent on ansible internals and the way that the task was invoked in the
|
||||
playbook. Note, that sometimes this can be quite large in size depending on the
|
||||
task and whether facts were enabled or not.
|
||||
|
||||
An example of a task event from the running system is::
|
||||
|
||||
{
|
||||
"status": "OK",
|
||||
"host": "puppetmaster.openstack.org",
|
||||
"session": "092aa3fa-2c73-11e7-bd24-bc764e048db9",
|
||||
"playbook_name": "compute*.ic.openstack.org:!disabled",
|
||||
"ansible_result": {
|
||||
"_ansible_parsed": true,
|
||||
"_ansible_no_log": false,
|
||||
"stdout": "",
|
||||
"changed": false,
|
||||
"stderr": "",
|
||||
"rc": 0,
|
||||
"invocation": {
|
||||
"module_name": "puppet",
|
||||
"module_args": {
|
||||
"logdest": "syslog",
|
||||
"execute": null,
|
||||
"facter_basename": "ansible",
|
||||
"tags": null,
|
||||
"puppetmaster": null,
|
||||
"show_diff": false,
|
||||
"certname": null,
|
||||
"manifest": "/opt/system-config/production/manifests/site.pp",
|
||||
"environment": "production",
|
||||
"debug": false,
|
||||
"noop": false,
|
||||
"timeout": "30m",
|
||||
"facts": null
|
||||
}
|
||||
},
|
||||
"stdout_lines": []
|
||||
},
|
||||
"ansible_type": "task",
|
||||
"ansible_task": "TASK: puppet : run puppet",
|
||||
"playbook_id": "b259ac6d-6cb5-4403-bb8d-0ff2131c3d7a",
|
||||
"ansible_host": "compute014.chocolate.ic.openstack.org"
|
||||
}
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user