Add documentation and tooling for legacy environments
A deployer may want to run these tools within a legacy environment (running Ansible <2.4) but will find it the deployment of these playbooks impossible to due to the use of new-ish task syntax, roles, and modules. This change gives deployers options when running within legacy environments by providing everything needed to deploy these playbooks using embeded ansible. Change-Id: Ic99b93017129321b2eb8b773a77f7fa478cc8dc7 Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This commit is contained in:
Kevin Carter
parent
dc5984c547
commit
4e0c30ed16
46
elk_metrics_6x/bootstrap-embeded-ansible.sh
Normal file
46
elk_metrics_6x/bootstrap-embeded-ansible.sh
Normal file
@ -0,0 +1,46 @@
|
||||
#!/usr/bin/env bash
|
||||
# Copyright 2018, Rackspace US, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
if [[ ! -f "/opt/ansible25/bin/ansible" ]]; then
|
||||
apt-get update
|
||||
apt-get -y install python3-virtualenv || apt-get -y install python-virtualenv
|
||||
virtualenv --python=/usr/bin/python3 /opt/ansible25 || virtualenv --python=/usr/bin/python2 /opt/ansible25
|
||||
/opt/ansible25/bin/pip install --upgrade ansible==2.5.2.0 --isolated
|
||||
fi
|
||||
|
||||
if [[ ! -d "/opt/ansible25/repositories/ansible-config_template" ]]; then
|
||||
mkdir -p /opt/ansible25/repositories
|
||||
git clone https://github.com/openstack/ansible-config_template /opt/ansible25/repositories/ansible-config_template
|
||||
fi
|
||||
|
||||
if [[ ! -d "/opt/ansible25/repositories/roles/systemd_service" ]]; then
|
||||
mkdir -p /opt/ansible25/repositories
|
||||
git clone https://github.com/openstack/ansible-role-systemd_service /opt/ansible25/repositories/roles/systemd_service
|
||||
fi
|
||||
|
||||
if [[ -f "/etc/openstack_deploy/openstack_inventory.json" ]]; then
|
||||
mkdir -p /opt/ansible25/inventrory
|
||||
cat > /opt/ansible25/inventrory/openstack_inventory.sh <<EOF
|
||||
#!/usr/bin/env bash
|
||||
cat /etc/openstack_deploy/openstack_inventory.json
|
||||
EOF
|
||||
chmod +x /opt/ansible25/inventrory/openstack_inventory.sh
|
||||
fi
|
||||
|
||||
export ANSIBLE_ROLES_PATH="/opt/ansible25/repositories/roles"
|
||||
export ANSIBLE_ACTION_PLUGINS="/opt/ansible25/repositories/ansible-config_template/action"
|
||||
export ANSIBLE_INVENTORY="${ANSIBLE_INVENTORY:-/opt/ansible25/inventrory/openstack_inventory.sh}"
|
||||
|
||||
export ANSIBLE_USER_VARS="$(for i in $(ls /etc/openstack_deploy/user_*secret*.yml); do echo -n "-e@$i "; done)"
|
||||
@ -2,13 +2,14 @@ Install ELK with beats to gather metrics
|
||||
########################################
|
||||
:tags: openstack, ansible
|
||||
|
||||
|
||||
About this repository
|
||||
---------------------
|
||||
|
||||
This set of playbooks will deploy elk cluster (Elasticsearch, Logstash, Kibana)
|
||||
with topbeat to gather metrics from hosts metrics to the ELK cluster.
|
||||
|
||||
**These playbooks require Ansible 2.4+.**
|
||||
**These playbooks require Ansible 2.5+.**
|
||||
|
||||
Before running these playbooks the ``systemd_service`` role is required and is
|
||||
used in community roles. If these playbooks are being run in an
|
||||
@ -20,6 +21,7 @@ ansible role requirements file.
|
||||
|
||||
git clone https://github.com/openstack/ansible-role-systemd_service /etc/ansible/roles/systemd_service
|
||||
|
||||
|
||||
OpenStack-Ansible Integration
|
||||
-----------------------------
|
||||
|
||||
@ -27,6 +29,7 @@ These playbooks can be used as standalone inventory or as an integrated part of
|
||||
an OpenStack-Ansible deployment. For a simple example of standalone inventory,
|
||||
see ``inventory.example.yml``.
|
||||
|
||||
|
||||
Optional | Load balancer VIP address
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
@ -36,11 +39,12 @@ provide the load balancer functionality needed. The option
|
||||
like Kibana will use when connecting to elasticsearch. If this option is
|
||||
omitted, the first node in the elasticsearch cluster will be used.
|
||||
|
||||
|
||||
Optional | configure haproxy endpoints
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Edit the `/etc/openstack_deploy/user_variables.yml` file and add fiel following
|
||||
lines
|
||||
Edit the `/etc/openstack_deploy/user_variables.yml` file and add the following
|
||||
lines.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
@ -74,8 +78,9 @@ Optional | run the haproxy-install playbook
|
||||
cd /opt/openstack-ansible/playbooks/
|
||||
openstack-ansible haproxy-install.yml --tags=haproxy-service-config
|
||||
|
||||
Deployment Process
|
||||
------------------
|
||||
|
||||
Setup | system configuration
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Clone the elk-osa repo
|
||||
|
||||
@ -112,38 +117,43 @@ Create the containers
|
||||
cd /opt/openstack-ansible/playbooks
|
||||
openstack-ansible lxc-containers-create.yml -e 'container_group=elastic-logstash:kibana'
|
||||
|
||||
install master/data elasticsearch nodes on the elastic-logstash containers
|
||||
|
||||
Deployment | legacy environment
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
If these playbooks are to be run in an environment that does not have access to
|
||||
modern Ansible source the script ``bootstrap-embeded-ansible.sh`` before running
|
||||
the playbooks. This script will install Ansible **2.5.2** in a virtual
|
||||
environment within ``/opt``. This will provide for everything needed to run
|
||||
these playbooks in an OpenStack-Ansible cloud without having to upgrade the
|
||||
Ansible version from within the legacy environment. When it comes time to
|
||||
execute these playbooks substite the ``openstack-ansible`` command with the
|
||||
full path to ``ansible-playbook`` within the embeded ansible virtual
|
||||
environment making sure to include the available user provided variables.
|
||||
|
||||
Example commands to deploy all of these playbooks using the embeded ansible.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd /opt/openstack-ansible-ops/elk_metrics_6x
|
||||
openstack-ansible installElastic.yml
|
||||
source bootstrap-embeded-ansible.sh
|
||||
/opt/ansible25/bin/ansible-playbook ${ANSIBLE_USER_VARS} site.yml
|
||||
|
||||
Install Logstash on all the elastic containers
|
||||
|
||||
Deploying | modern environment
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Install master/data elasticsearch nodes on the elastic-logstash containers,
|
||||
deploy logstash, deploy kibana, and then deploy all of the service beats.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd /opt/openstack-ansible-ops/elk_metrics_6x
|
||||
openstack-ansible installLogstash.yml
|
||||
|
||||
Install Kibana, nginx reverse proxy and metricbeat on the kibana container
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd /opt/openstack-ansible-ops/elk_metrics_6x
|
||||
openstack-ansible installKibana.yml
|
||||
|
||||
Install Metricbeat everywhere to start shipping metrics to our logstash
|
||||
instances
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd /opt/openstack-ansible-ops/elk_metrics_6x
|
||||
openstack-ansible installMetricbeat.yml
|
||||
openstack-ansible site.yml
|
||||
|
||||
|
||||
Adding Grafana visualizations
|
||||
-----------------------------
|
||||
Optional | add Grafana visualizations
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
See the grafana directory for more information on how to deploy grafana. Once
|
||||
When deploying grafana, source the variable file from ELK in order to
|
||||
@ -152,11 +162,12 @@ dashboards. Including the variable file is as simple as adding
|
||||
``-e @../elk_metrics_6x/vars/variables.yml`` to the grafana playbook
|
||||
run.
|
||||
|
||||
Included dashboards
|
||||
Included dashboards.
|
||||
|
||||
* https://grafana.com/dashboards/5569
|
||||
* https://grafana.com/dashboards/5566
|
||||
|
||||
|
||||
Trouble shooting
|
||||
----------------
|
||||
|
||||
|
||||
@ -20,73 +20,73 @@ logstash_beat_input_port: 5044
|
||||
|
||||
# Beat options
|
||||
heartbeat_services:
|
||||
- group: "{{ groups['galera_all'] }}"
|
||||
- group: "{{ groups['galera_all'] | default([]) }}"
|
||||
name: galera HTTP
|
||||
ports:
|
||||
- 9200
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/"
|
||||
- group: "{{ groups['galera_all'] }}"
|
||||
- group: "{{ groups['galera_all'] | default([]) }}"
|
||||
name: galera TCP
|
||||
ports:
|
||||
- 3306
|
||||
type: tcp
|
||||
- group: "{{ groups['repo_all'] }}"
|
||||
- group: "{{ groups['repo_all'] | default([]) }}"
|
||||
name: repo git
|
||||
ports:
|
||||
- 9418
|
||||
type: tcp
|
||||
- group: "{{ groups['repo_all'] }}"
|
||||
- group: "{{ groups['repo_all'] | default([]) }}"
|
||||
name: repo server
|
||||
ports:
|
||||
- 8181
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/"
|
||||
- group: "{{ groups['repo_all'] }}"
|
||||
- group: "{{ groups['repo_all'] | default([]) }}"
|
||||
name: repo acng
|
||||
ports:
|
||||
- 3142
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/acng-report.html"
|
||||
- group: "{{ groups['glance_api'] }}"
|
||||
- group: "{{ groups['glance_api'] | default([]) }}"
|
||||
name: glance api
|
||||
ports:
|
||||
- 9292
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/healthcheck"
|
||||
- group: "{{ groups['glance_api'] }}"
|
||||
- group: "{{ groups['glance_api'] | default([]) }}"
|
||||
name: glance registry
|
||||
ports:
|
||||
- 9191
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/healthcheck"
|
||||
- group: "{{ groups['gnocchi_all'] }}"
|
||||
- group: "{{ groups['gnocchi_all'] | default([]) }}"
|
||||
name: gnocchi api
|
||||
ports:
|
||||
- 8041
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/healthcheck"
|
||||
- group: "{{ groups['heat_api_cfn'] }}"
|
||||
- group: "{{ groups['heat_api_cfn'] | default([]) }}"
|
||||
name: heat cfn api
|
||||
ports:
|
||||
- 8000
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/"
|
||||
- group: "{{ groups['heat_api'] }}"
|
||||
- group: "{{ groups['heat_api'] | default([]) }}"
|
||||
name: heat api
|
||||
ports:
|
||||
- 8004
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/"
|
||||
- group: "{{ groups['keystone_all'] }}"
|
||||
- group: "{{ groups['keystone_all'] | default([]) }}"
|
||||
name: keystone api
|
||||
ports:
|
||||
- 5000
|
||||
@ -94,49 +94,49 @@ heartbeat_services:
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/"
|
||||
- group: "{{ groups['neutron_server'] }}"
|
||||
- group: "{{ groups['neutron_server'] | default([]) }}"
|
||||
name: neutron server
|
||||
ports:
|
||||
- 9696
|
||||
type: http
|
||||
method: GET
|
||||
path: "/"
|
||||
- group: "{{ groups['nova_api_metadata'] }}"
|
||||
- group: "{{ groups['nova_api_metadata'] | default([]) }}"
|
||||
name: nova api metadata
|
||||
ports:
|
||||
- 8775
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/"
|
||||
- group: "{{ groups['nova_api_os_compute'] }}"
|
||||
- group: "{{ groups['nova_api_os_compute'] | default([]) }}"
|
||||
name: nova api compute
|
||||
ports:
|
||||
- 8774
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/"
|
||||
- group: "{{ groups['nova_api_placement'] }}"
|
||||
- group: "{{ groups['nova_api_placement'] | default([]) }}"
|
||||
name: nova api placement
|
||||
ports:
|
||||
- 8780
|
||||
type: http
|
||||
method: GET
|
||||
path: "/"
|
||||
- group: "{{ groups['nova_console'] }}"
|
||||
- group: "{{ groups['nova_console'] | default([]) }}"
|
||||
name: nova console
|
||||
ports:
|
||||
- 6080
|
||||
- 6082
|
||||
- 6083
|
||||
type: tcp
|
||||
- group: "{{ groups['cinder_api'] }}"
|
||||
- group: "{{ groups['cinder_api'] | default([]) }}"
|
||||
name: cinder api
|
||||
ports:
|
||||
- 8776
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/"
|
||||
- group: "{{ groups['horizon_all'] }}"
|
||||
- group: "{{ groups['horizon_all'] | default([]) }}"
|
||||
name: horizon
|
||||
ports:
|
||||
- 80
|
||||
@ -144,96 +144,96 @@ heartbeat_services:
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/"
|
||||
- group: "{{ groups['sahara_api'] }}"
|
||||
- group: "{{ groups['sahara_api'] | default([]) }}"
|
||||
name: sahara api
|
||||
ports:
|
||||
- 8386
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/healthcheck"
|
||||
- group: "{{ groups['swift_proxy'] }}"
|
||||
- group: "{{ groups['swift_proxy'] | default([]) }}"
|
||||
name: swift proxy
|
||||
ports:
|
||||
- 8080
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/healthcheck"
|
||||
- group: "{{ groups['aodh_api'] }}"
|
||||
- group: "{{ groups['aodh_api'] | default([]) }}"
|
||||
name: aodh api
|
||||
ports:
|
||||
- 8042
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/"
|
||||
- group: "{{ groups['ironic_api'] }}"
|
||||
- group: "{{ groups['ironic_api'] | default([]) }}"
|
||||
name: ironic api
|
||||
ports:
|
||||
- 6385
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/"
|
||||
- group: "{{ groups['rabbitmq_all'] }}"
|
||||
- group: "{{ groups['rabbitmq_all'] | default([]) }}"
|
||||
name: rabbitmq management
|
||||
ports:
|
||||
- 15672
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/"
|
||||
- group: "{{ groups['rabbitmq_all'] }}"
|
||||
- group: "{{ groups['rabbitmq_all'] | default([]) }}"
|
||||
name: rabbitmq access
|
||||
ports:
|
||||
- 5672
|
||||
- 5671
|
||||
type: tcp
|
||||
- group: "{{ groups['magnum_all'] }}"
|
||||
- group: "{{ groups['magnum_all'] | default([]) }}"
|
||||
name: magnum api
|
||||
ports:
|
||||
- 9511
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/"
|
||||
- group: "{{ groups['trove_api'] }}"
|
||||
- group: "{{ groups['trove_api'] | default([]) }}"
|
||||
name: trove api
|
||||
ports:
|
||||
- 8779
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/"
|
||||
- group: "{{ groups['barbican_api'] }}"
|
||||
- group: "{{ groups['barbican_api'] | default([]) }}"
|
||||
name: barbican api
|
||||
ports:
|
||||
- 9311
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/"
|
||||
- group: "{{ groups['designate_api'] }}"
|
||||
- group: "{{ groups['designate_api'] | default([]) }}"
|
||||
name: designate api
|
||||
ports:
|
||||
- 9001
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/"
|
||||
- group: "{{ groups['octavia_all'] }}"
|
||||
- group: "{{ groups['octavia_all'] | default([]) }}"
|
||||
name: octavia api
|
||||
ports:
|
||||
- 9876
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/"
|
||||
- group: "{{ groups['tacker_all'] }}"
|
||||
- group: "{{ groups['tacker_all'] | default([]) }}"
|
||||
name: tracker api
|
||||
ports:
|
||||
- 9890
|
||||
type: http
|
||||
method: HEAD
|
||||
path: "/"
|
||||
- group: "{{ groups['neutron_server'] }}"
|
||||
- group: "{{ groups['neutron_server'] | default([]) }}"
|
||||
name: opendaylight
|
||||
ports:
|
||||
- 8180
|
||||
- 8185
|
||||
type: tcp
|
||||
- group: "{{ groups['neutron_server'] }}"
|
||||
- group: "{{ groups['neutron_server'] | default([]) }}"
|
||||
name: ceph-rgw
|
||||
ports:
|
||||
- 7980
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user