32 Commits

Author SHA1 Message Date
Jesse Pretorius
af73200d70 MNAIO: Correct README regarding DATA_DISK_DEVICE
The DATA_DISK_DEVICE environment variable no longer exists.
In this patch we correct the README with the appropriate
instructions for changing the data disk device.

Change-Id: I2790b6d848a2960d89f122b0f5bb1cf714c68809
2018-09-20 12:38:53 +01:00
Dave Wilde
525887872b Deploy ELK in MNAIO
This enables the deployment of the elk_metrics_6x stack inside of an
MNAIO.

Change-Id: Ie611baee79c33d7cbab9f0865127ac5966475838
2018-09-11 20:17:24 +00:00
Zuul
26d0d53ea5 Merge "Adds iPXE UEFI support to Multi Node AIO" 2018-08-27 21:38:48 +00:00
Jesse Pretorius
484059205a MNAIO: Enable using a data disk for file-backed VM's
In order to make use of a data disk, we enable the 'file'
implementation of default_vm_disk_mode to use a data disk
much like the 'lvm' implementation.

To simplify changing from the default_vm_disk_mode of lvm
to file and back again, the setup-host playbook will remove
any previous implementation and replace it. This is useful
when doing testing for these different modes because it
does not require cleaning up by hand.

This patch also fixes the implementation of the virt
storage pool. Currently the tasks only execute if
'virt_data_volume.pools is not defined', but it is always
defined so the tasks never execute. We now ensure that
for both backing stores the 'default' storage pool is
defined, started and set to auto start (as three tasks
because the virt_pool module sucks really bad and can only
do one thing at a time).

The pool implementation for the 'file' backed VM's uses
the largest data disk it can find and creates the /data
mount for it. To cater for a different configuration, we
ensure that all references to the disk files use the path
that is configured in the pool ,rather than assuming the
path.

Change-Id: If7e7e37df4d7c0ebe9d003e5b5b97811d41eff22
2018-08-15 16:58:50 +01:00
Antony Messerli
fec5aa8f20 Adds tweaks to run MNAIO on Ubuntu 18.04
Drops python-software-properties package as it
doesn't appear to be in the Bionic build.

Change-Id: I70ee08b69374e7e61a6ac3344a93f119748941e9
2018-07-23 14:10:45 -05:00
Jesse Pretorius
7e94c86084 MNAIO: Correct Mac OS X instructions for virt-manager
After various attempts to make virt-manager work on a Mac
using brew and other methods, I found a way that's simpler
and much lighter on the client. This patch changes the
README to reflect this method.

Change-Id: I4c7594e7c0c371b4c0e417a78c8833262e479d22
2018-06-26 11:54:16 +01:00
Jesse Pretorius
329aa472f2 MNAIO: Enable saving and re-using file-based backing images
Being able to save the images and re-use them on other hosts
is extremely useful to cut down deployment time. This patch
allows an MNAIO setup to be setup using a file-based backing
store, then have those saved and re-used on the same host or
on other hosts.

Change-Id: I491d04fb94352e37312891a9b9bd58093fdd00cf
2018-06-26 11:54:16 +01:00
Jesse Pretorius
67c0746b71 MNAIO: Update README to include using the file-based backing store
Using the file-based backing store is currently undocumented. This
patch updates the README to include instructions for using it.

Change-Id: I228171b6619512874aece46f705ee1a922610cd5
2018-06-26 11:54:16 +01:00
Jesse Pretorius
1e845e8d20 MNAIO: Correct README
The README is currently misleading and outdated. This corrects
a few things related to rebuilding the test environment.

Change-Id: I6c9b1698fb77ddfcb4c7ade6cd5a7a14a14c55e6
2018-06-26 11:54:16 +01:00
Antony Messerli
02542eaaf8 Make OSA repo location configurable
Adds a variable for specifying the OSA repo to check out.

Change-Id: I7f87c59f16cea0c4acbfb02d83a4ec3db9df2a45
2018-06-07 17:25:50 -05:00
Antony Messerli
bc611698c8 Adds iPXE UEFI support to Multi Node AIO
Now expects a base URL that looks for iPXE files.  Uses
boot.ipxe.org by default but the base URL can be overridden to
point to custom compiled versions of ipxe.lkrn, ipxe.efi,
and undionly.kpxe.

Change-Id: Ib2c10c4817eaf9a78326f3de00aed0c16f8df776
2018-03-26 13:29:06 -05:00
Antony Messerli
f00c6bf215 Replace pxelinux with iPXE support in MNAIO
Adds support for iPXE scripting.  Allows for the
operator to specify a remote iPXE kernel to pull in
newer iPXE kernel features and specify a remote
default iPXE script for customizing the multi node
for other types of deployments.

Switches the installations to use netboot installer kernels
instead of ISO retrival and extraction.

Restructures directory to allow for future multi node configs
instead of just debian based installs.

Change-Id: Ie97f021dbd33ad3d852132b0b2850f54d4e13476
2018-02-07 18:20:00 -06:00
Kevin Carter
7fbc50c036
Add env vars to set the RAM for VMS in the mnaio
The MNAIO serves as a good way to play with a production like OSA
environment on minimal hardware. While this is a good way to test it
does leave some options hard to address and change when trying to build
in an automated way. This change makes it possible for a deployer to set
an environmental variable to change the ram allocation of all default vm
types which should help folks build a more capable and customizable test
cloud.

Change-Id: Idc2cff091701ecda80165a98fffa1db222600d27
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-01-03 17:14:56 -06:00
Antony Messerli
e3a1d1fcb9 Allows for the VMs preseed mirror to be overridden
In the case the upstream mirror might be down or there might be a
faster mirror available, this allows you to specify the desired
Ubuntu mirror and base directory.

Change-Id: I49c33b9e4e107990f2abb418b859fd280ba91107
2017-12-13 16:39:06 -06:00
Antony Messerli
c92d4c5938 Update readme to use force flag instead
echo y seems to still prompt for each volume
when using -f will complete the action without
prompting.

Change-Id: Ic8231817656e2093e53e0bfcd1fd5e72bd8181b2
2017-10-16 15:59:40 -05:00
Kevin Carter
cc9294bb0d
use linux-image-generic by default
The package, linux-image-generic, should be used by the MNAIO to ensure
we're testing with the supported LTS kernel for the distro being
deployed.

Change-Id: I25b3318a02a3c8e79fb300e93b2fdb18a7cc3b20
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-10-10 01:03:16 -05:00
Kevin Carter
be1d365ef5
allow a default kernel to be set
Change-Id: I6cf0cf918df6c79757b283333f5a687ff20a46bd
2017-10-09 22:28:16 -05:00
Kevin Carter
3da22ca992 Allow for the use of an external inventory
It makes sense to allow a user to provide their own inventory as needed.
this change simply allows a user to explort the option "MNAIO_INVENTORY"
prior to running `build.sh` which will allow them to define their own
inventory using the general purpose build script.

Change-Id: Ib5233da1bf34620ec8bf3decc4ecd51823b50b79
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-09-27 10:08:07 -05:00
Matt Thompson
28684e6c6e Add missing features to multi-node-aio
The multi-node-aio update that moved the provisioning from bash to
ansible dropped a few features that we use for gating purposes.  This
commit re-adds the following:

1. The ability to drop iptables rules to do port redirection from the
   host to private IPs.  This is controlled by CONFIG_PREROUTING and
   the ansible variable mnaio_host_iptables_prerouting_ports.
2. /etc/hosts on the physical node is now updated w/ the hostname and
   IP of each VM so we can access VMs by name.

NOTE: With #1, we redirect to the VM's DHCP address, and not it's
      management address.  The latter seemed to the desired address
      but didn't work, which is why we've resorted to DHCP.  If using
      this address is incorrect please note so we can investigate
      further.

Change-Id: Ib194c314280f2474a2e4dac6d0feba44b1ee696f
2017-09-13 11:47:25 -04:00
Stuart Grace
f4594e9107 Add setup of OpenStack with flavors, images, etc
Added playbook which replicates the actions of openstack-service-setup.sh
which was present in earlier versions. This creates a sample set of
flavors, Linux images, networks, security group rules and a router so
OpenStack is ready to use.

The data controlling what is created is all in the file
   multi-node-aio/playbooks/vars/openstack-service-config.yml

Change-Id: Ib1999f215aabadb23a3ebeb55fbce4a2caf69030
2017-08-17 15:12:31 +01:00
Kevin Carter
369f68832e
Add environment options and re-flow the README.rst
Change-Id: I7a2640856045e36043de8508f9421fbd8a593591
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-08-01 09:13:15 -05:00
Kevin Carter
cfc76ded4a
Convert vars in files to host_vars
This change allows the MNAIO to really be used as a stand alone kick
system which has the potential to be developed into a stand alone
project. At the very least this change improves playbook performance
by scoping variables.

The inventory has been converted into a typical Ansible inventory and
the "servers" used in the MNAIO are now simply host_vars
which will trigger specific VM builds when instructed to do so. This
gives the MNAIO the ability to serve as a stand alone kick system which
could be used for physical hosts as well as MNAIO testing all through
the same basic set of playbooks. Should a deployer want to use this with
physical servers they'd need to do nothing more than define their basic
inventory and where the the required pieces of infrastructure needed to
PXE boot their machines.

Change-Id: I6c47e02ecfbe8ee7533e77b11041785db485a1a9
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-07-31 23:31:13 -05:00
Kevin Carter
a94f0a9026 Combine our two multi-node-aio processes into one
The original mnaio was built using a lot of bash and was tailored
specifically for ubuntu 14.04. The new mnaio was built using a mix of
bash and ansible and was tailored specifically for ubuntu 16.04. This
patch takes the two code bases and combines the best things from each
method and wraps it up into a single code path all written using ansible
playbooks and basic variables.

While underlying system has changed the bash environment variable syntax
for overrides remains the same. This allows users to continue with what
has become their normal work-flow while leveraging the new structure and
capabilities.

High level overview:
  * The general performance of the VMs running within the MNAIO will now
    be a lot better. Before the VMs were built within QCOW2 containers,
    while this was flexible and portable it was slower. The new
    capabilities will use RAW logical volumes and native IO.
  * New repo management starts with preseeds and allows the user to pin
    to specific repositories without having to worry about flipping them
    post build.
  * CPU overhead will be a lot less. The old VM system used an
    un-reasonable number of processors per VM which directly translated
    to sockets. The new system will use cores and a single socket
    allowing for generally better VM performance with a lot less
    overhead and resource contention on the host.
  * Memory consumption has been greatly reduced. Each VM is now
    following the memory restrictions we'd find in the gate, as a MAX.
    Most of the VMs are using 1 - 2 GiB of RAM which should be more than
    enough for our purposes.

Overall the deployment process is simpler and more flexible and will
work on both trusty and xenial out of the box with the hope to bring
centos7 and suse into the fold some time in the future.

Change-Id: Idc8924452c481b08fd3b9362efa32d10d1b8f707
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-07-28 15:35:23 +00:00
Melissa Kam
0b581feae3 Add option to just configure prerouting rules
Change-Id: Ia297cc98201388231a0a4c4338790dd5734bb432
2017-04-25 14:31:48 -05:00
Weezer Su
0df9b3dab1 Add kernel option into preseed
According to suggestion, add the kernel option in the preseed file,
leave it empty to get the lastest kernel, or specify the kernel
number.

Change-Id: Ie5b8f62a89ef9eb51954f0452becbaa9e81ee973
2017-02-08 10:24:14 -06:00
Weezer Su
7daef611b9 Add instruction of the option for Operation system for VMs
Add the DEFAULT_IMAGE instruction into README.rst

Change-Id: I77e1b43a2533638af50d7430195f6162763126cd
2017-01-18 13:51:14 -06:00
Weezer Su
ff9002ba36 Add PRE_CONFIG_OSA option and move the RUN_OSA to the run playbook part
Add PRE_CONFIG_OSA to set up the user_variable and all the other
parameters for OSA powered cloud, and move the RUN_OSA to the playbook
part.

Change-Id: Ia962e42466dd2a123e30b1a9c37e60a371710842
2016-11-02 11:53:40 -05:00
Weezer Su
c8f1af0bcf Remove the expired key and opensuse repo after cobbler being installed
Remove the opensuse expired key and opensuse cobbler repo after
cobbler being installed. OSA deployment doesn't need it.

And fix a typo and a link of README.rst

Change-Id: I24f5d823806f039d879dd69762b2eac3a6005628
2016-10-20 16:13:46 -05:00
Jenkins
1b4cb5699a Merge "Add remove known_hosts in the instruction for Multi-Node AIO" 2016-10-20 19:14:44 +00:00
Weezer Su
22501ca02a Add remove known_hosts in the instruction for Multi-Node AIO
Add the rm ~/.ssh/known_hosts for the Rerunning the build script
part in the instruction.

Change-Id: Iba7610e00f07bb61cd8b11892e208ea4c67a1d75
Closes-Bug: #1635037
2016-10-19 16:04:28 -05:00
Weezer Su
9b5320dcb9 Add an option for deploying OSA or other OSA powered cloud
Add an option $RUN_OSA to the deploy-osa.sh to let user have more
choices, if user want to deploy another OSA powered cloud on AIO-
multinode, he can switch it to false and then use the existing env
to deploy the cloud he wants to.

Change-Id: Ief8c701f596595607d3b86ac03f17d6690f5954f
Closes-Bug: #1634582
2016-10-18 16:38:00 -05:00
Kevin Carter
a7180a096c
Ported the osa-multi-node-aio
This change imports the code from [ https://github.com/cloudnull/osa-multi-node-aio ]
so that others in the OSA community can benifit from and contribute to
the project.

Change-Id: Id9a2d928f7e8bbd185725349e8ce51344f19daa4
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-08-10 09:40:40 -05:00