300 Commits

Author SHA1 Message Date
Jenkins
005ca3d90e Merge "remove wait_for from glance-api" 2015-04-21 20:53:47 +00:00
Daneyon Hansen
17fc2d9b47 Removes existing network namespaces for neutron-agents container
Previously, network namespaces would not work correctly when
restarting or stop/rm/start the neutron-agents container. This
is because network namespaces created within the container are
associated to the container's pid: /pid/$CONTAINER_PID/ns/net.
With this patch, when a container starts or restarts, the l3/dhcp
agent start scripts will remove any existing qrouter/qdhcp network
namespaces before starting the agents. The agents will recreate the
necessary network namespaces since they are stored in the db.

Closes-Bug: 1444219

Change-Id: Ia86729766fe8c2fc145b3a02d519746b149a73bb
2015-04-21 07:50:48 +00:00
Steven Dake
64a2cf40a5 Add workaround to work with cirros image
The cirros image uses an old dhcp client which does not work
properly with hardware that has checksum offload enabled.  Add
a bootp rule to rewrite the checksum via iptables.

Latest cirros 0.3.33 is afflicted by this bug.

Change-Id: Ibfd1f87af5d0bef9fcb1354121e76e4e0db8ab5e
2015-04-15 19:18:41 -07:00
Steven Dake
095fbd7d7c Neutron doa in current form
The nova-config.sh script was exiting because of incorrect shell syntax.

The correct syntax for a regex search is
[[ haystack =~ needle ]].  The .*needle.* is unnecessary.  I think this shell
script exited because a period was missing.

Change-Id: I3aca5e0729eb5fedbe9ecb4d75ed85d5a1c7815a
2015-04-14 18:55:48 -07:00
Steven Dake
f8768acff7 remove wait_for from glance-api
For some reason glance sometimes fails to permanently find keystone
while other services do find keystone.  The host also has full access
to keystone.  Change wait_for to fail_unless.

This could be a docker bug, a wait_for bug, or some other problem.

Change-Id: I02d611d65b7ffddb9c27101fd60e2a8b7cc25658
2015-04-14 11:20:03 -07:00
Steven Dake
f4319a9a21 CentOS 7.1 image doesn't work with libvirt
The libvirt package has a dependency on systemd-libs and systemd,
while the centos 7.1 container installs systemd-container-libs
and systemd-container.  We don't need either of these systemd tools
but they reuslt in a broken build of the system if they are missing.

As is, the libvirt container will not build.

Change-Id: I8e5aac0e3a7e9d88a81733a3048971fe9d48ba56
2015-04-14 10:56:41 -07:00
Daneyon Hansen
882a859ff5 Removes check_required_vars for Log Files
Previously, check_required_vars was requiring log file variables
to be set in openstack.env. This would cause an error when
starting nova/neutron containers that used default (blank) log
file settings. Since the log files are not required
for the services to start, the default log files are left blank
(without a value in the K/V pair). This patch removes the
log_file variable for check_required_vars in each nova/neutron
script.

Change-Id: I169baab28f63a0e0ae06c8aef21b5e8d3155bdda
2015-04-13 17:17:52 +00:00
Daneyon Hansen
ed107a5e26 Adds Neutron Agents Support
Previously, the agents ran in seperate containers. Since some of
the agents create and use network namespaces, runnning them in
seperate containers has become a challenge. This patch adds
neutron agent support in a single container.

Partially Implements: blueprint compute-operation-neutron

Change-Id: If4f893abd50d90b58e8435ec2659870dce36951c
2015-04-13 07:34:12 +00:00
Daneyon Hansen
2f2bb366e0 Adds neutron-server to neutron container-set
Adds neutron-server components of the neutron container set.

Partially Implements: blueprint compute-operation-neutron

Change-Id: I63525ae85f4ead4e010cdb9b6bc8059bb9d252e7
2015-04-10 03:11:23 +00:00
Daneyon Hansen
0b671ad58d Adds Initial Logging Support
Previously, logging was hard coded into the nova and neutron
containers. This patch creates two common params for logging:

DEBUG_LOGGING
VERBOSE_LOGGING

and sets default values of verbose=true and debug=false. This patch
implements logging for Nova and Neutron. Add'l OS service config
scripts and images will need to be updated to use the logging
params.

Change-Id: I03d48005d5b4789aa3c519879a20d963ef1cbaa3
2015-04-07 19:13:29 +00:00
Jenkins
5ea37557cf Merge "Remove building of pam+noaudit and place in copr instead" 2015-04-07 14:53:52 +00:00
Jenkins
7a29958949 Merge "libvirtd provides was removed from centos7.1" 2015-04-07 14:50:47 +00:00
Steven Dake
a87980e5c2 Remove EXPOSE options from Dockerfiles
The EXPOSE options will create a local docker-proxy.  This is
unnecessary with --net=host mode.  The docker-proxy adds about
20 microseconds of latency.  Add documentation to the specification
to indicate where to find the ports that are exposed by the
services in case someone were to desire to add EXPOSE back to
the Dockerfiles.

Change-Id: I398e922fe096d6022a2d5985bb92498f89a5ea31
2015-04-06 20:25:18 -07:00
Steven Dake
f4506ddd60 Remove building of pam+noaudit and place in copr instead
COPR is a repository for third party packages built by the Fedora build
system.  Instead of dragging in 250MB of dependencies to build pam, build
externally and load only the RPM into the image.

The pam that is built externally is built with the --with-noaudit flag
and a Provides: pam line.

Change-Id: I28396996943d2fdc12b253faf62af8623c8b0e73
2015-04-06 20:22:34 -07:00
Steven Dake
1a530dca4c libvirtd provides was removed from centos7.1
the libvirtd package "Provides" wwas removed from the libvirt packaging.
To workaround this problem just install libvirt-daemon in CentOS 7.1 or Fedora
20+.

WIthout this change, an install of libvirtd does not actually install
libvirtd.  The yum tool will not fail if libvirtd is specified so we weren't
catching this in the build.

Change-Id: I81e64f6f659aa0466369dad4b0574ebb24c99555
2015-04-06 11:09:19 -07:00
Jenkins
0573da4424 Merge "Load appropriate modules for libvirt" 2015-04-03 18:01:15 +00:00
Jenkins
ba175c61e1 Merge "Simplify wait_for calls" 2015-04-02 13:53:06 +00:00
Jenkins
92b129968d Merge "Deal with CentOS 7.1 breakage" 2015-04-02 01:06:40 +00:00
Steven Dake
82db9eb6c7 Deal with CentOS 7.1 breakage
The baseurl is busted in CentOS latest RDO images

Change-Id: Id619a28a6207a8c70835a12416e6f7b432d49811
2015-04-01 16:00:14 -07:00
Sam Yaple
2f0f37f97b Load appropriate modules for libvirt
Loads ebtable_nat, ip6_tables, ip_tables, and the appropriate kvm module
if the cpu supports it.

Change-Id: I69995b06da6da4169a2f0d207608543820d8a89e
2015-04-01 08:35:34 -05:00
Martin André
056e02d0d3 Simplify wait_for calls
Rely on the the `check_for_*` functions and remove redundant
`fail_unless_*` calls.

Also change `wait_for` to exit when it is missing a required argument.

Change-Id: I90c4545691d53185556e2838303ac3df0afaf9fa
2015-04-01 16:41:52 +09:00
Martin André
34872c1c2f Allow check_for_* functions to be consumed by wait_for
In order for the `check_for_*` functions to be consumed by `wait_for`,
they should notify of their success but not exit.

As a consequence, the previous behavior is restored by the fail_unless_*
companion functions.

With this change, it is now possible to do:

    wait_for 30 1 check_for_os_service_running keystone

Change-Id: I16ddf8913027030c3ccb5487713d172904508fd6
2015-04-01 16:39:44 +09:00
Steven Dake
93e164d973 Test for presence of /dev/kvm before setting permissions
In a native QEMU environment there is no /dev/kvm.

Change-Id: I4d0eb32fd6fad3a4c9c15c7c2b12430b2e4db154
2015-03-31 00:12:42 +00:00
Steven Dake
fe380064bf Add libvirt-driver-lxc to nova-libvirt
libvirt is compiled with lxc support.  On Ubuntu hosts, without this file
libvirt is nonusable.  The container exits immediately.

Change-Id: Ifbc2d75defe082cbe3b80b1821799f1f965c2a85
2015-03-30 06:57:03 -07:00
Jenkins
62dbf832ad Merge "Partially Implements: Container set compute-operation-neutron" 2015-03-28 20:27:08 +00:00
Sam Yaple
986edf4a97 Replace ; with && in Dockerfile
When using ';' this can allow the previous command to fail and while the
docker build proceeds without realizing a command failed. Switching to
'&&' allows the exit code to make it to the docker build command and the
build to fail appropriately.

Change-Id: Idd0991ed4549542bb10d27da1a0a025d0503b6c1
2015-03-27 18:52:48 -05:00
Daneyon Hansen
40be5646fa Partially Implements: Container set compute-operation-neutron
Adds the neutron-linuxbridge-agent to the neutron container set.

Change-Id: I20752611c24a3110cc5d2e4fca2d501fc32a4d87
2015-03-27 23:36:06 +00:00
Steven Dake
afcc1569b8 Work around Ubuntu 14.04 Kernel bug with --net=host
This patch builds pam with no auditing.  This allows the images to
operate on Ubuntu 14.04 which contain a kernel bug and have not been
updated.

Change-Id: I2b8bd192abcc26df27ca276dc3d0fde377048f20
2015-03-26 13:39:16 -07:00
Daneyon Hansen
1182e060cb Extends Nova Metadata Service Parameterization and Composition
Previously, nova enabled_apis was hard coded and metdata_host
was using PUBLIC_IP param instead of exposing an individual
param.

Extending these params and creating a separate compose yml that
includes compute/libvirt/network/api is required to support
nova network multi_host.

Additional details can be found here:

http://docs.openstack.org/admin-guide-cloud/content/section_metadata-service.html

Change-Id: I5d417a0ee460f8ad9b2982fe83d63ab57013918e
2015-03-25 17:27:44 +00:00
Daneyon Hansen
81d65f0b3d Refactor nova-network
Previously, eth0 was set as the nova flat interface. This
interface is used to bridge VM traffic between instances and
should not have an IP. It's a problem when you associate an
interface with an IP to the FLAT_INTERFACE.

Change-Id: I38c516094430e2d55b1b4c047a1bbcfe711a2dff
2015-03-24 18:44:28 +00:00
Jenkins
d944259056 Merge "Fixes bug where heat container was not getting heat meta data" 2015-03-23 20:48:41 +00:00
Ryan Hallisey
1aa0f7ce49 Fixes bug where heat container was not getting heat meta data
Heat container is having trouble accessing heat meta data.
Heat needs to have HEAT_CFN_API_SERVICE_HOST set in order
to properly receive meta data.

Closes-bug 1435288

Change-Id: Ia28ece3ae91ec7acc62cc86039612841ac487f36
2015-03-23 16:22:05 -04:00
Steven Dake
c2b92c5e96 Make mariadb use a data container without a bindmount
A data container is meant to operate without bindmounts.

Change-Id: I2624a63fd75db7ed8050054c17e19885f8a7ac8a
Co-Authored-By: Daneyon Hansen (danehans@cisco.com)
2015-03-22 08:55:24 -07:00
Steven Dake
f4fd47dc2c Make nova-compute-network use a data volume container
A data volume container is far superior to bind mounting the
host's shared directories.  It preserves the idempotency,
immutability and declarative properties of the containers.

The way this works in practice is that a data volume container is
created.  Then when the containers start they use LVM to access
the filesystem where /var/lib/docker is contained.  Then the
container startup logic bindmounts the data volume stored on the
host filesystem in /var/lib/docker/vfs/dir/ID.  This prevents
people with access to the host operating system from damaging
the contents of the data container.

It does mean that now we must use tools/stop to stop our containers
rather than tools/cleanup-containers -f.

This is a containers best practice.  For more details see:

https://docs.docker.com/userguide/dockervolumes/

Big credit goes to Danyeon Hansen for seeding this idea in the
mariadb containers.

Note occasionally docker-compose start/stop seems to not want to stop
a container.  This bug needs to be addressed upstream separately
from our utilization of this best practice.

Change-Id: Iaa1419f606e1b1b7a7560a095c49e79d643164f1
2015-03-22 08:54:53 -07:00
Jenkins
be79b50b31 Merge "Don't kill keystone, but instead wait for it to finish at the end" 2015-03-21 03:19:48 +00:00
Jenkins
f84c2b1705 Merge "Fix wait_for function in kolla-common.sh" 2015-03-21 03:17:15 +00:00
Jenkins
ad6026e0e1 Merge "Refactors nwfilter" 2015-03-21 03:16:24 +00:00
Jenkins
68978c2338 Merge "Make the Horizon dashboard work properly" 2015-03-21 03:15:58 +00:00
Jenkins
b6d4cf6310 Merge "Cleanup leftovers after wait_for was merged into kolla-common.sh" 2015-03-21 03:07:48 +00:00
Jenkins
17eccb0dfd Merge "Implements: blueprint Implement a database container set" 2015-03-21 02:47:52 +00:00
Martin André
448e48a75b Cleanup leftovers after wait_for was merged into kolla-common.sh
Change-Id: I97ab671bfbdcf5434feedec617a938b30af4579b
2015-03-21 11:11:57 +09:00
Daneyon Hansen
c49613ad0a Refactors nwfilter
Previously, the nwfilter daemon/conf pkgs would need to be
installed on the host. Compose would then mount the host DIR
to the compute containers. A more efficient approach is to simply
install the nwfilter config/daemon in the libvirt container. Both
appoaches address the following error when booting an instance:

'no-mac-spoofing' is missing

Change-Id: I81b446d198920473bf99b97ef5eac6ef5ee85024
2015-03-20 22:00:54 +00:00
Steven Dake
f642fd7b2d Don't kill keystone, but instead wait for it to finish at the end
use the bash wait builtin to wait for the keystone process to exit.

Change-Id: I03057e7925e7e41120c163c0be22ade75050d9f1
2015-03-20 10:48:32 -07:00
Steven Dake
71576f4e42 Make the Horizon dashboard work properly
The Horizon dashboard was not working properly prior to this work.

Also a docker-compose file was added to start the dashboard.

Change-Id: I8a438c15b967b0cec00bfe44b997f833bf745191
2015-03-20 10:21:29 -07:00
Daneyon Hansen
5547b5fe78 Implements: blueprint Implement a database container set
Previously, the database container was configured for use with
Kubernetes. This patch removed any k8s dependencies, adds a script
to manage mysql server.cnf settings and splits data and app
containers. Splitting the containers provides additional
portability and operational efficiencies compared to host mounts.

Change-Id: I80656450c02dda5f2959d187eec20d5877dc54a2
2015-03-20 16:17:51 +00:00
Daneyon Hansen
8eb36e7cde Fixes Keystone openrc and sets / for scripts
Previously, the keydtone openrc file was not using the proper
auth url, whcih was broken with too many double ticks. Also
changed the auth port from public to admin since keystone user
is an admin tenant. Changed DIR for scripts to align with the
rest of the project.

Change-Id: Iadde3239227e65ecca479b16a7b7db51d3a579c8
2015-03-20 07:18:21 +00:00
Martin André
1cb5514353 Fix wait_for function in kolla-common.sh
Add `wait_for_output` and `wait_for_output_unless` helper functions that
only succeed when the command output contains the expected string, with
a possibility to exit early with a failure for `wait_for_output_unless`.

Change-Id: Ie20e7c72fb84f626f1a3fe9c4d2f22c9d6863dbe
2015-03-20 12:55:53 +09:00
Steven Dake
858cdda3ec Use keystone authentication with Glance
Previously glance was not using keystone for authentication.  This resulted
in --is-public true failing when doing image-create.

Co-Authored-By: Lars Stedman (larsks@redhat.com)

Change-Id: I54273115242ee169201643fe0ad559392650c883
2015-03-19 09:21:48 -07:00
Jenkins
3309f5e4a0 Merge "Make rabbitmq container work with it's proper hostname set" 2015-03-19 13:34:13 +00:00
Steven Dake
38fada551c mariadb-server requires the hostname userspace command
The hostname command is required to allow mariadb to access
the network properly.  Without it, in some cases mariadb cannot
execute name service resolution.

Change-Id: Ia215585faad23012143b63c4d82acd6ba69f59b7
2015-03-18 20:07:36 -07:00