The current setup was using processor cores from ansible facts which in
a multi-core, single socket system could result in 1. Using the
processor count will return the logical processor count giving us a more
performant setup when the compute power is present.
Change-Id: Ia5b63d45691f58e848d05cc4a4e5f353b993a347
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The KVM collector would attempt to use a network connection which
normally results in failure. This changes the collector to use a file
system socket by default.
Change-Id: Id1698a95644c6a6d5102e371a7266794196393c8
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
When we save the VM disks, we now use compression to prepare
the base disk so that it reduces the base disk file sizes to
a much smaller base. For the infra hosts this reduces the file
from ~23GB to ~8GB. Once this is done, we then also create a
copy-on-write image in the original disk's place so that the
VM can be booted up to verify functionality or continue work
without having to re-run the 'deploy-vms' playbook.
Change-Id: If95b71d8625b4d5b2a036cec13952e4fd73cecd4
Using the discard option for all mount points ensures
that the deletes actually release the blocks on the disk.
This ensures that SSD performance is optimised and that
file-backed images are kept as small as possible.
Change-Id: I648cbaca56d75e355cf6c8af01e2e3ad20dfc398
When using file-backed storage, or SSD storage, any erasing
done in the VM does not actually clear up the space. By using
the virtio-scsi controller the VM is able to use TRIM to clear
any blocks which are deleted.
This also allows us to use fstrim to reduce the size of the
qemu files before we save them for later re-use.
Change-Id: Ia9001522ce054ee9f8a6dd38270da3e3fd039813
When switching from LVM to a file backing store, the existing pool
cannot be undefined until any existing VM's and LV's are removed.
This ensures that if this is the case, it will be cleaned up so
that the switch is effortless.
Change-Id: Ie1460b37593306044f0a63f445c3da1987362d34
Instead of putting the images in the root of the disk,
we use a subdirectory. This prevents silly mistakes
from happening.
Change-Id: I19d22b7e72de88736db410a771ec22664c641c94
This is a mechanically generated patch to complete step 1 of moving
the zuul job settings out of project-config and into each project
repository.
Because there will be a separate patch on each branch, the branch
specifiers for branch-specific jobs have been removed.
Because this patch is generated by a script, there may be some
cosmetic changes to the layout of the YAML file(s) as the contents are
normalized.
See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html
Change-Id: I03144a0d6e8654ce841072554750f2ec7aa646f6
Story: #2002586
Task: #24319
When executing the save, the return code of 2 indicates
a change was implemented - but I forgot to include that
condition for failed_when. As it stands now the task will
be considered failed which is a bit useless.
Change-Id: Ie8b36335048d2dcf6d0f9e66f8440430f4a68398
When deploying metricbeat there are some container environments that
will have a mount point within the container that may differ from that
of the host; like in the case of an LXC container with an LVM backed
bind mount or an NFS share. This change will now check if the filesystem
within a container has any mounts and compare the return to that of the
provided physical host if the list is >0 the fs checks will be deployed
within the contianer environment.
Change-Id: Iae5827f4e7e0a85eb733128b54d6ef4c8721537a
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
When not using a file-backed backing store, the vriable is not
defined and results in an error to that effect.
Change-Id: I3142a5960bc4521f79bbdfe32b0e7a0f71742b7d
CI testing is experiencing intermittent failures when deploying base glance
images as part of the openstack-image-setup.yml playbook which is kicked of
as part of the openstack-service-setup.yml playbook in openstack-ansible-ops.
Since the deployment of these resources rely on external URI endpoints, this
type of failure is something that can occur during a customer deployment.
Change-Id: Ieea0f11482646ea152920a1ff1009a2b03705f1c
The TESTING_BRANCH environment variable is provided by the
run_tests.sh script and is derived from the .gitreview file.
This ensures that once the master branch becomes a stable
branch, the constraints from the stable branch in the
integrated repository will automatically get used once the
.gitreview file is updated.
To ensure that the required environment variables are present
we export them appropriately in run_tests.sh and modify the
tox configuration to pass them into the tox test.
Change-Id: I4a101851d656e5dce3a2cc9eb5d5508b46953c4b
Needed-By: https://review.openstack.org/579371
When running on a system where there is no journal or no systemd skip
the journalbeat playbooks.
Change-Id: I92c804e8eb2ab2f9b86eca09fc51d19be66c7190
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The embedded bootstrap process can support all of our OS's, this change
ensures thats possible.
Change-Id: I730bb775aa5e9f87609ea885142d7361203cbb2c
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The basic index template will ensure that the replication policy is
enforced on the revolving indexes we know will exist.
Change-Id: I1e3edcfd00a73cbdd328d50e8ba6492ac2248b72
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
To allow us to use the json_query filter, we ensure that
the jmespath distro package is installed onto the host.
Change-Id: Icb9053fd3a7486030f4336130fe6ad503852b07a
In order to more successfully reproduce an environment using
saved images, we include the VM XML definition files and the
output from 'pip freeze'. We capture the list of files, their
checksums and the SHA for the git repo into a json manifest
file.
Change-Id: Ia0bf74d509b4acb10b0dd832a4cfe1bb2afb2503
The config `default_bind_mount_logs` was being set in the leap tooling
which results in all containers not having a log bind mount. This change
removes the task that sets the offending config.
Change-Id: Ib85e77ed1f993bad44f1cb36ed11056f8a83dfc5
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
It's better to shut the VM's down cleanly instead of just turning
them off, so we change 'destroy' to 'shutdown' and use the virt
module for this action instead of the command module.
Change-Id: I896b7794328b91dc59726bf1d5366eeb7112ca21
In order to make use of a data disk, we enable the 'file'
implementation of default_vm_disk_mode to use a data disk
much like the 'lvm' implementation.
To simplify changing from the default_vm_disk_mode of lvm
to file and back again, the setup-host playbook will remove
any previous implementation and replace it. This is useful
when doing testing for these different modes because it
does not require cleaning up by hand.
This patch also fixes the implementation of the virt
storage pool. Currently the tasks only execute if
'virt_data_volume.pools is not defined', but it is always
defined so the tasks never execute. We now ensure that
for both backing stores the 'default' storage pool is
defined, started and set to auto start (as three tasks
because the virt_pool module sucks really bad and can only
do one thing at a time).
The pool implementation for the 'file' backed VM's uses
the largest data disk it can find and creates the /data
mount for it. To cater for a different configuration, we
ensure that all references to the disk files use the path
that is configured in the pool ,rather than assuming the
path.
Change-Id: If7e7e37df4d7c0ebe9d003e5b5b97811d41eff22
An operator might want to ensure Swift is available during
an upgrade and manually upgrade Swift on completion of the
maintenance.
The operator would need to set these vars before operation:
export SKIP_SWIFT_UPGRADE=yes
export CONTAINERS_TO_DESTROY=add_!swift_all_exclusion
This would prevent the swift containers from being torn
down during the upgrade and would skip all Swift upgrade
operations.
Change-Id: Ibf40499750751dd9f41e447b7b90bb77f592cc14
Rather than installing pip packages on to the host system,
we can just execute the script and it will use the ansible
runtime venv. This works for Ocata onwards. Any earlier
releases can either pre-install the right packages, or
implement the change to the script shebang in a fork.
Change-Id: I88eb4c1bc9fe3a38803c5f0f5d1725dbed74dac7
There is already a default in group_vars/all, so we do not need
to provide a default in every conditional.
Also, we move several LVM data volume tasks into a block given
they have a common set of conditions.
Change-Id: Iff0fafefda2bc5dc1596b7198b779f5da763086c
Now that the roll-up has been implemented the original shrink method is
no longer required or useful. This change cleans up things up.
Change-Id: I24fd5b4daafc2f48ee5a3421f6b58b157a7aff6c
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Given there is almost no difference between the releases, we can
use the same vars file and simple conditionals. The package
'software-properties-common' is available for Trusty & Xenial so
we just use that and remove the unnecessary extra package.
We also now add the correct UCA repositories for Trusty and Bionic
so that we get the latest version of libvirt.
Finally, we simplify the conditional for the iptables binary to
make it far easier to read.
Change-Id: Id4b3711a4d7a0ccc13db956d41017ac01c97825f
