Will Szumski 956a29f83a Support customizing prometheus.cfg files

This provides a mechanism to scrape targets defined outside of kolla-ansible.

Depends-On: https://review.opendev.org/#/c/685671/
Change-Id: I0950341b147bb374b4128f09f807ef5a756f5dfa
Related: blueprint custom-prometheus-targets

2020-05-11 13:47:12 +01:00

2.5 KiB

Raw Blame History

Prometheus - Monitoring System & Time Series Database

Overview

Kolla can deploy a full working Prometheus setup in either a all-in-one or multinode setup.

Preparation and deployment

To enable Prometheus, modify the configuration file /etc/kolla/globals.yml and change the following:

enable_prometheus: "yes"

Extending the default command line options

It is possible to extend the default command line options for Prometheus by using a custom variable. As an example, to set remote timeouts to 30 seconds and data retention period to 2 days:

prometheus_cmdline_extras: "-storage.remote.timeout 30s -storage.local.retention 48h"

Extending prometheus.cfg

If you want to add extra targets to scrape, you can extend the default prometheus.yml config file by placing additional configs in {{ node_custom_config }}/prometheus/prometheus.yml.d. These should have the same format as prometheus.yml. These additional configs are merged so that any list items are extended. For example, if using the default value for node_custom_config, you could add additional targets to scape by defining /etc/kolla/config/prometheus/prometheus.yml.d/10-custom.yml containing the following:

scrape_configs:
  - job_name: custom
    static_configs:
      - targets:
        - '10.0.0.111:1234'
  - job_name: custom-template
    static_configs:
      - targets:
{% for host in groups['prometheus'] %}
        - '{{ hostvars[host]['ansible_' + hostvars[host]['api_interface']]['ipv4']['address'] }}:{{ 3456 }}'
{% endfor %}

The jobs, custom, and custom_template would be appended to the default list of scrape_configs in the final prometheus.yml. To customize on a per host basis, files can also be placed in {{ node_custom_config }}/prometheus/<inventory_hostname>/prometheus.yml.d where, inventory_hostname is one of the hosts in your inventory. These will be merged with any files in {{ node_custom_config }}/prometheus/prometheus.yml.d, so in order to override a list value instead of extending it, you will need to make sure that no files in {{ node_custom_config }}/prometheus/prometheus.yml.d set a key with an equivalent hierarchical path.

2.5 KiB Raw Blame History