84 Commits

Author SHA1 Message Date
Matt Thompson
33fb2d2ffa Use separate interfaces for lb traffic
Currently, deploying a multi-node AIO results in
internal_lb_vip_address and external_lb_vip_address being set to the
same IP (10.0.236.150), which results in services having issues
communicating with the lb.

This commit simply sets external_lb_vip_address to use the
load balancer's dhcp address, which is fixed to 10.0.2.150.

Change-Id: I6faabd641c0559a0381e559f364cc76c49293014
2017-08-29 19:46:25 +00:00
Matt Thompson
e3eb50bb16 Create /root/.tmux.conf if it does not exist
Currently, we use lineinfile against /root/.tmux.conf, however if the
file does not first exist the deploy-osa.yml playbook will fail.  This
commit simply updates the `Config Tmux` task by adding the create option
to lineinfile so that the config will get created if it doesn't first
exist.

Change-Id: I366c4a875c6e7c2482f28a46bb29a2882b773f30
2017-08-28 15:04:44 -04:00
Jenkins
abb648fa34 Merge "Pass deployment_environment_variables to OSA" 2017-08-22 17:55:37 +00:00
Stuart Grace
27531d4a97 Pass deployment_environment_variables to OSA
When using environment variables to setup a proxy, it checks whether
global_environment_variables (for permanent proxy) or
deployment_environment_variables (for playbooks only) is in use and
sets up matching user_variables for OSA. Previously, it only setup
global environment variables.

Added | bool filters as suggested in review.

Change-Id: I6554cd28f817ab0d128a1a469a6007b043851c07
2017-08-22 14:48:04 +00:00
Jenkins
ac4d431fdc Merge "General improvements" 2017-08-18 04:29:41 +00:00
Weezer Su
99bc655fb9 Add Tmux configuration on infra1
add tmux configuration on infra1 to raise up the history-limit to
20000, which helps troubleshooting the deployment.

Change-Id: I075364868c0b74a297ad6b8ca7dbfec71ca91c27
2017-08-17 11:46:51 -05:00
Kevin Carter
67161e5d81
General improvements
Adding even more dynamic user config options and a compute preseed.

Change-Id: Ieccbd61a45a77b7dff9e228fc6fdaf92df852abb
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-08-17 09:16:20 -05:00
Kevin Carter
ebcfc0d0a4
General improvements
Adding more dynamic user config options.

Change-Id: I6f30ea38e3680ffa2ef1a77eadc91cfa9d946971
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-08-15 16:55:48 -05:00
Kevin Carter
c678e83275
General improvements
* added ip alias for interfaces

* Update settings and improve vm performance

* Change the VG name in VMs. The VG name was changed so that the volume
  which is being used by VMs can be mounted on a physical host, and not
  conflict, with standard volume group naming. This is usful when a VM
  is DOA and a deployer wants to disect the instance.

Change-Id: If4d10165fe08f82400772ca88f8490b01bad5cf8
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-08-14 10:35:12 -05:00
Kevin Carter
eb5debbdbb
move mnaio network group var because it's shared
Change-Id: I3c85564def49f3d0da76c55dc4356cbf16a25d63
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-08-10 19:13:14 -05:00
Kevin Carter
6dbba8a3f8
fix vm server ip typo and isolate hosts into ip blocks
Change-Id: I27d4b9a304241c7572f3c634fb7d3ef4eaed6a3e
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-08-01 13:14:24 -05:00
Kevin Carter
369f68832e
Add environment options and re-flow the README.rst
Change-Id: I7a2640856045e36043de8508f9421fbd8a593591
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-08-01 09:13:15 -05:00
Kevin Carter
cec9c6bebc
Add force handlers to the build command
This change ensures that all handlers that are triggered are fired which
should assist in cases where a playbook fails and is then rerun.

Change-Id: I6b53611df72cbd0a98bc6d58cff786ae3662e9c5
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-08-01 09:03:25 -05:00
Jenkins
850c763a19 Merge "Convert vars in files to host_vars" 2017-08-01 13:36:49 +00:00
Antony Messerli
505f5bb19a Ensure openstack_deploy directory is create before copy
Change-Id: Ic28bbd01f0647914223885c89c7843b20073e081
2017-08-01 10:48:51 +00:00
Kevin Carter
cfc76ded4a
Convert vars in files to host_vars
This change allows the MNAIO to really be used as a stand alone kick
system which has the potential to be developed into a stand alone
project. At the very least this change improves playbook performance
by scoping variables.

The inventory has been converted into a typical Ansible inventory and
the "servers" used in the MNAIO are now simply host_vars
which will trigger specific VM builds when instructed to do so. This
gives the MNAIO the ability to serve as a stand alone kick system which
could be used for physical hosts as well as MNAIO testing all through
the same basic set of playbooks. Should a deployer want to use this with
physical servers they'd need to do nothing more than define their basic
inventory and where the the required pieces of infrastructure needed to
PXE boot their machines.

Change-Id: I6c47e02ecfbe8ee7533e77b11041785db485a1a9
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-07-31 23:31:13 -05:00
Kevin Carter
a3d39c43cb
fix typo in osa user config for swift and inets
Change-Id: I0befea6c7e403ca847721de3a5a8338e0e734098
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-07-31 17:55:28 -05:00
Kevin Carter
9abaeba8c8
move deploy node to infra1 and make a LB node
Sadly the log node does not have enough ram to run a full ansible run.
Ansible 2.x requires more ram than one would expect, especially when the
inventory gets large. this change moves the deploy node to infra1 as it
will already have the ram needed to run the playbooks. Additionally the
container storage for infra nodes was too small which forces builds into
error. The default storage for VMs has been set to 90GiB each and the
preseed will create a logical volume for VMs mounted at /var/lib/lxc.

While the limited ram works well for the VMs and within a running
deployment of OSA, ansible-playbook is subject to crash like so:

  An exception occurred during task execution. To see the full traceback,
  use -vvv. The error was: OSError: [Errno 12] Cannot allocate memory
  fatal: [infra1_cinder_api_container-b38b47ea]: FAILED! =>
  {"failed": true, "msg": "Unexpected failure during module execution.", "stdout": ""}

So infra nodes have had the memory constraint raised to 8GiB

Change-Id: I7175ea92f663dfef5966532cfc0b4beaadb9eb03
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-07-29 15:52:36 -05:00
Kevin Carter
3f17e2b5db
add acng to the host system and preseeds
This change ensures that the VMs and host systems cache apt packges
locally which will speed up the boot and deployment process.

Change-Id: I234e338b9f1b9f11ff1e81ede8c5717e033fdad8
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-07-28 18:53:57 -05:00
Kevin Carter
3788abcf8b
correct CPU topology
Change-Id: Ica5306483991a893a4bfd4f1aba8ad00cd51d045
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-07-28 17:02:34 -05:00
Kevin Carter
c3800224b0
move drive layout to deploy-vms and fix deploy-osa tags/tasks
Change-Id: Iaee4c3683d798320099dec77286e6fac7a10bee8
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-07-28 14:49:27 -05:00
Kevin Carter
ca24b28fdc
re-added missing bindep
Change-Id: Ia23b9096a5af10290e3aee368a69205151fdc7fd
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-07-28 11:47:33 -05:00
Kevin Carter
96e9384277 Remove the deploy vm and use the log node instead
This reduces the resource consumption by removing the deploy node and
using the log node instead. This also ups the ram allocation to the
infra hosts which will improve the deployment experience by ensuring we
don't run out of memory.

Change-Id: Id38ff386669308ac3fd1e539ae37c969f00353b8
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-07-28 15:36:49 +00:00
Kevin Carter
7917eb144d load virtio kernel modules by default
Change-Id: Ibef9b6ff273784566e6c7b479cacd2d2dd97fe0d
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-07-28 15:36:44 +00:00
Kevin Carter
a94f0a9026 Combine our two multi-node-aio processes into one
The original mnaio was built using a lot of bash and was tailored
specifically for ubuntu 14.04. The new mnaio was built using a mix of
bash and ansible and was tailored specifically for ubuntu 16.04. This
patch takes the two code bases and combines the best things from each
method and wraps it up into a single code path all written using ansible
playbooks and basic variables.

While underlying system has changed the bash environment variable syntax
for overrides remains the same. This allows users to continue with what
has become their normal work-flow while leveraging the new structure and
capabilities.

High level overview:
  * The general performance of the VMs running within the MNAIO will now
    be a lot better. Before the VMs were built within QCOW2 containers,
    while this was flexible and portable it was slower. The new
    capabilities will use RAW logical volumes and native IO.
  * New repo management starts with preseeds and allows the user to pin
    to specific repositories without having to worry about flipping them
    post build.
  * CPU overhead will be a lot less. The old VM system used an
    un-reasonable number of processors per VM which directly translated
    to sockets. The new system will use cores and a single socket
    allowing for generally better VM performance with a lot less
    overhead and resource contention on the host.
  * Memory consumption has been greatly reduced. Each VM is now
    following the memory restrictions we'd find in the gate, as a MAX.
    Most of the VMs are using 1 - 2 GiB of RAM which should be more than
    enough for our purposes.

Overall the deployment process is simpler and more flexible and will
work on both trusty and xenial out of the box with the hope to bring
centos7 and suse into the fold some time in the future.

Change-Id: Idc8924452c481b08fd3b9362efa32d10d1b8f707
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-07-28 15:35:23 +00:00
Melissa Kam
8a182302ee Support older versions for getting service ports
Kilo and Liberty have a different name for the
haproxy services than newer versions.

Change-Id: I070eb22cc219451f42c436fa31e4db33fa79a8fe
2017-07-07 11:19:11 -05:00
Weezer Su
8c98c87885 Fix a typo in all seed files
the auto reboot config should be
d-i finish-install/reboot_in_progress note
not
finish-install finish-install/reboot_in_progress note

Change-Id: Id8e44de5e6376bb576115b0eeca49dc04edf8ed8
2017-06-30 16:58:31 -05:00
Melissa Kam
8d92f0515b Increase size of deploy node to speed up deployments
Change-Id: I735b8f714ecead21df24cc3543616647e6b1ab94
2017-06-28 11:50:35 -05:00
Melissa Kam
700c878da5 Reduce range of tempest public subnet
The current range can cause tempest tests to
allocate floating IPs to the created test servers
that conflict with the IPs of the host VMs.

Change-Id: Ib885122f0ee291f10ecfe9ef4735adc9ae5f5cbf
2017-06-22 13:13:40 -05:00
Matthew Thode
a22b573223
make services persistant across reboots
also switch to systemctl commands

Change-Id: I26c7dec1512f88be4faa8800199cf8e92dc678ba
2017-05-24 11:57:37 -05:00
Weezer Su
eb5349bc78 Add the public ip to iptables rules
Adding the ip to the filter to make the DNAT request rightly.

Change-Id: I5e229444bd873cad59f5d038b82294c1f6acd853
2017-04-26 14:02:08 +00:00
Melissa Kam
0b581feae3 Add option to just configure prerouting rules
Change-Id: Ia297cc98201388231a0a4c4338790dd5734bb432
2017-04-25 14:31:48 -05:00
Melissa Kam
3dcb39b79d Fix variable exports for deployment
The newline was preventing some of the variables from
being exported properly.

Change-Id: If189f6e288796d828da749c5873983ea46764068
2017-04-21 16:16:56 -05:00
Weezer Su
06f1368490 Add deploy node to multinode-aio enviroment.
Add deploy node into the multinode-aio env, move the deployment
and HAproxy into it, and NAT the port 22, 80, 443 into the deploy
node.

Change-Id: I7c886f213e6b3398cb4ecbc36a4ee55fb7f7fd10
2017-04-19 16:57:06 -05:00
Kevin Carter
1e5c0c5cb9
define the default kernel statically
Change-Id: I6d39460de826f73e0ae28e0feb6a173d384583a3
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-03-16 19:24:00 -05:00
Weezer Su
9a48d3728d Fix the kernel version been deployed on VMs
Base on the http://packages.ubuntu.com/trusty/kernel/, the
linux-image-generic will install the 3.13.0.112.120 [amd64, i386]
kernel. leave it empty will do nothing to the kernel. this is
important when you deploy ubuntu 14.04.5 on VMs which use 4.4 rather
then 3.13.

Change-Id: I7984edaef5a65320ec3b10085646e8ad5c93f65d
2017-03-15 19:37:57 +00:00
Weezer Su
2a34b74d56 Remove the NETWORK_BASE from build.sh
remove the NETWORK_BASE from build.sh because it already been setup
at the variables.sh

Change-Id: I5143691ea83e9c8f8c7f91e6a56788fa62b4eee4
2017-03-09 15:56:32 -06:00
Nolan Brubaker
f93ec3d2be Allow environment variable overrides
As is, the variables.sh file will clobber envvars. Don't do that
anymore.

Change-Id: I6afc0eb354c19ebc96fc51f0dcf06d89aa3cabfe
2017-03-08 16:26:08 -05:00
Jenkins
a31dc59f17 Merge "Allow unset varaibles when checking http_proxy" 2017-02-23 14:28:21 +00:00
Melissa Kam
fac7844a32 Allow unset varaibles when checking http_proxy
The build.sh script uses set -eu, which causes this script to
fail if http_proxy is not set.

Change-Id: I5a71106db13fcd7c7c5fe5bc3350441e556759c1
2017-02-22 14:59:44 -06:00
Jenkins
9d9580d091 Merge "Add kernel option into preseed" 2017-02-22 19:31:10 +00:00
Melissa Kam
83201edfb5 Remove whitespaces before EOF
Change-Id: I6023839bf10e6b22cf41e7022b736cf9e7387f93
2017-02-21 16:07:44 -06:00
Jenkins
90d1757eaf Merge "Adds more values into variables.sh and places in extra scripts" 2017-02-20 19:57:54 +00:00
Jenkins
439dc5180c Merge "Adds in option to disable overriding of apt-sources list" 2017-02-20 19:57:48 +00:00
Jenkins
4c53790432 Merge "Adds in option to modify DNS - removes hardcoded values" 2017-02-20 16:16:32 +00:00
Rick Box
550b2824cc Adds more values into variables.sh and places in extra scripts
More values have been added to the variables.sh file. None of these have
been adjusted from the default values but this now provides a standard
goto place to have all these values and be able to change with ease.

An include for varibles.sh is now also at the top of each of the script
files.

Change-Id: I2354c87e985115ce0b7f168ab1d056d5032e5cff
2017-02-20 16:07:44 +00:00
Rick Box
d38b7b059a Adds in option to disable overriding of apt-sources list
By default AIO deploy overrides apt-sources, if things like a local
mirror are already set up then this script will override these. This
option allows for the override to be disabled.

Change-Id: Id84c2c5d12153125d7ac6698493c41a840993c5b
2017-02-20 15:32:15 +00:00
Rick Box
d261d1a6e8 Adds in option if proxy is set - push through to containers
This commit allows for a proxy to be pushed to each of the deployed
containers based on the standard environment variable "$http_proxy".

Change-Id: I94dc3a8afda9472f032fbd94991f8b676131cbfb
2017-02-20 15:16:05 +00:00
Rick Box
3baca05501 Adds in option to modify DNS - removes hardcoded values
Some networks may not allow 8.8.8.8 Google DNS servers. This patch
allows this to be specified in the variables file or to pull the value
from the systems resolv.conf file.

Change-Id: I494ab82c41bcd59ea85404d49cb2696029353a0e
2017-02-20 12:49:46 +00:00
Melissa Kam
1b40ea4cf3 Use Trusty sources for any 14.04 build
Change-Id: I42b2a63b899cd1254669569dee04f0821cb08ba0
2017-02-17 09:55:09 -06:00