218 Commits

Author SHA1 Message Date
Monty Taylor
15663daaf7 Add iptables role
Co-Authored-By: James E. Blair <corvus@inaugust.com>
Change-Id: Id8b347483affd710759f9b225bfadb3ce851333c
Depends-On: https://review.openstack.org/596503
2018-08-27 14:33:32 +00:00
Zuul
18bcfeb5c8 Merge "Don't create "admin" group" 2018-08-24 19:57:29 +00:00
Zuul
fcfacc7e52 Merge "Add system-config-run-base job" 2018-08-24 19:57:27 +00:00
Ian Wienand
114953cbff Don't create "admin" group
According to the Ubuntu 12.04 release notes, up until Ubuntu 11.10
admin access was granted via the "admin" unix group, but was changed
to the "sudo" group to be more consistent with Debian et al.

Remove the now unnecessary group

Modify the install-ansible role to set some directory ownership to
root:root; there didn't seem to be any reason to use admin here.

This means the "users" role is no longer required in the bridge.yaml,
as it is run from the base playbook anyway.

Change-Id: I6a7fdd460fb472f0d3468eb080aebbb010931e11
2018-08-24 08:50:22 -07:00
James E. Blair
19aebca3e7 Add system-config-run-base job
This adds a job which creates a bridge-like node and bootstraps it,
and then runs the base playbook against all of the node types we
use in our control plane.  It uses testinfra to validate the results.

Change-Id: Ibdbaf511bbdaee46e1335f2c83b95ba1553a1d94
Depends-On: https://review.openstack.org/595905
2018-08-24 08:50:01 -07:00
Zuul
36386394ca Merge "Allow bridge playbook to run under zuul" 2018-08-24 04:34:51 +00:00
James E. Blair
1fa136cba9 Allow bridge playbook to run under zuul
Normally the bridge playbook runs as root on bridge.  In order to
allow zuul to bootstrap a bridge-like node in its tests while running
as the zuul user, add become: true to the playbook.  This will have
no effect on bridge itself, but will cause the playbook to behave
in the same manner in tests.

Also add the "users" role to bridge.  This is in the base playbook
and is therefore eventually run on bridge.  However it needs to also
be in the bridge playbook in order to bootstrap bridge correctly, as
the install-ansible role references groups which are created in the
users role.

Change-Id: If311914e9e632d8be855fff0a62528dd191bf1d0
2018-08-23 08:29:01 -07:00
Ian Wienand
1939f3e8ec Move exim role to top-level
Move the exim role to be a "generic" role in the top-level roles/
directory, making it available for use as a Zuul role.

Update the linters jobs to look for roles in the top level

Update the Role documentation to explain what the split in roles is
about.

Change-Id: I6b49d2a4b120141b3c99f5f1e28c410da12d9dc3
2018-08-23 21:34:47 +10:00
Ian Wienand
4402201d56 roles: Add README.rst and lint
These role docs aren't exactly War and Peace, but I think longer term
as we fiddle about making things generic or not and moving them
around, we'll be better off having kept ourselves to writing
*something*.

Add terse README.rst files for all existing roles, and add simple
linter check to ensure new roles get them too.

Change-Id: Ibc836310fb8a45e12c2e31f112d92509ac350413
2018-08-23 21:34:42 +10:00
Ian Wienand
564a17812f playbooks/roles/exim: remove "true" filter plugin
This filter is unsued in the role, remove it.

This allows it to be run under zuul and can be moved into the
top-level role/ directory later.

Change-Id: Ice97f0c3c9f52b6bf9f48c7b16d577e555924034
2018-08-23 14:49:50 +10:00
Ian Wienand
b99c4c6fb0 Generate roles documentation
Since we're building out roles in system-config now, generate
documentation.  We look in roles/* and playbook/roles/* (follow-on
changes will split things up between the two).

Correct the reference names in the exim documentation to avoid
warnings and failure.

This also revealed a single unicode character in the exim readme
(which caused prior versions of zuul-sphinx to barf).  For fun, see if
you can find it!

Depends-On: https://review.openstack.org/#/c/579474/
Change-Id: I243a96bbd6d09560f8aa80b6345b90039422547a
2018-08-23 14:49:50 +10:00
Monty Taylor
de7fbba912
Set path in cron so we can find ansible-playbook
ansible-playbook is in /usr/local/bin which is not in the path in cron.

Change-Id: I633c021936dfbbfdb022aa3d052fe88f32d2ef6a
2018-08-21 16:55:42 -05:00
Monty Taylor
759015f101
Start running puppet cron on bridge.openstack.org
Puppet cron is no longer being run on puppetmaster (yay!) so start
running it in cron from bridge.

Change-Id: Idc579a2660a5450092544c21a2e9e6cb9688e5f9
2018-08-20 18:26:30 -05:00
Monty Taylor
1b54971705 Ensure apt is used on ubuntu hosts with zypper
There is an issue with our nb0* hosts where they have zypper installed
for building suse images but that tricks ansible in to thinking it
should use zypper for package management.

This has been submitted upstream as
https://github.com/ansible/ansible/pull/44413

Change-Id: I96f60501e43bfe9c6acb4ce80f8450b245943ca8
2018-08-20 20:45:13 +00:00
Zuul
ac3d9d4651 Merge "Add local copy of pkg_mgr fact module" 2018-08-20 20:41:04 +00:00
Zuul
45930d6cb4 Merge "Have ansible retry ssh connections" 2018-08-20 20:41:03 +00:00
Zuul
24c86f1648 Merge "Manage sshd config with ansible" 2018-08-20 20:41:00 +00:00
Zuul
744d7fb737 Merge "Remove infracloud references" 2018-08-20 20:21:40 +00:00
Monty Taylor
9911799b76
Add local copy of pkg_mgr fact module
We need to override this. This is the upstream version unmodified.

Change-Id: Iec3a22595a2f1f802922c22e883c4d63859c3886
2018-08-20 13:18:21 -05:00
Monty Taylor
f6533076f7
Have ansible retry ssh connections
In zuul's ansible config we add retries=3 to deal with transient issues.
Do the same thing for our production runs.

Change-Id: Ide53bae34e5e622de1fd4741706752e8728da20e
2018-08-20 13:18:21 -05:00
Monty Taylor
eae9a42b4b
Manage sshd config with ansible
We write out a specific sshd config file. Let's do it with ansible.

Change-Id: Ie92b6ec6c8772d31009d1c2a2f7d5558bb38f67a
2018-08-20 11:26:38 -05:00
Monty Taylor
0625c289c8
Remove infracloud references
We don't run a cloud anymore and don't use these. With the cfg
management update effort, it's unlikely we'd use them in the form they
are in even if we did get more hardware and decide to run a cloud again.

Remove them for clarity.

Change-Id: I88f58fc7f2768ad60c5387eb775a340cac2c822a
2018-08-20 11:03:55 -05:00
Zuul
2011113c3f Merge "Install ssh key on bridge.openstack.org" 2018-08-19 16:18:53 +00:00
Monty Taylor
c4b111691b
Install ssh key on bridge.openstack.org
We copied this over from puppetmaster, but let's manage it in ansible.

The key has been renamed in host_vars on bridge.openstack.org already.

Change-Id: Ia102dbe2ae2836880092b8997cb99135f5197b00
2018-08-19 10:26:10 -05:00
Monty Taylor
fee717fcbd
Use is changed instead of | changed
Using tests as filters is going away. Use it as a test.

Change-Id: Icc60395186c950d50e539592dcabc772dda96c8a
2018-08-17 16:05:26 -05:00
Monty Taylor
01a90aadb3
Add loop_control and loop_var to silence a warning
The CentOS tasks run inside of a loop in tasks/main.yaml. That means
that item has been defined in the loop there. While it's currently
working, go ahead and add loop_control.loop_var to remove the clash.

Change-Id: I0e8288c35645945aa9b43fb02c29576c1ad31d7e
2018-08-17 15:47:26 -05:00
Monty Taylor
5f6c56998e
Add skip: true to disable-puppet-agent
We don't need to do anything on RH as services don't start by default
there.

Change-Id: I38daee0b80006711cb0c4c65aed108ac0c5c215a
2018-08-17 15:38:41 -05:00
Monty Taylor
83ebf61e07
Be more explicit about puppet paths
puppet wants the code to be in /opt/system-config/production because of
the environment config. bridge just wants /opt/system-config because
it's an ansible server.

Rather than relying on inferring things, just be explicit about what we
want where.

Depends-On: https://review.openstack.org/593134
Change-Id: I9e749d2c50f7d8a7b0681fe48f38f4741c8a8d01
2018-08-17 14:26:22 -05:00
Monty Taylor
a634593a05
Set mgmt_hieradata in puppet group_vars
This is not a variable describing the system-under-management
bridge.openstack.org - it's a variable that is always true for all
systems in the puppet group.

As a result, update the puppet apply test to figure out which directory
we should be copying modules _from_ - since the puppet4 tests will be
unhappy otherwise.

Change-Id: Iddee83944bd85f69acf4fcfde83dc70304386baf
2018-08-17 14:25:50 -05:00
Monty Taylor
dfcd9c5403
Be explicit about where to install roles
The first entry is where ansible galaxy will install roles. We want that
to be /etc/ansible/roles, not overlaid on the system-config repo.

Pass --roles-path to ansible-galaxy to make sure they go to the right
place.

Change-Id: I109dc004acad32a515c6a1caca50ab38edc62aed
2018-08-17 13:09:59 -05:00
Monty Taylor
03f52d0bc8
Move ansible log into directory
file: state=touch returns changed every time. Instead, put the log files
into a /var/log/ansible directory.

Change-Id: I086d803f0e532b9da41cb01d4e7d2ed66245dfc1
2018-08-17 12:57:20 -05:00
Monty Taylor
ee622108a5
Remove restricted and multiverse from ubuntu servers
restricted is supported software that is non-free.
multiverse is unsupported software that is non-free.

Use of software from either would be unacceptable on any Infra server,
so remove them from the sources.list files.

While we're in there, clean things up a little bit and add an arm file
for bionic.

Change-Id: I55a3b3d411e8a3496a4e6910baaf72f3c192e9d4
2018-08-17 11:53:52 -05:00
Monty Taylor
5380eb5b6e
Remove purge_apt_sources
This was a setting added for infra cloud that had to do with bootstrap
order. It seems to have been cargo-culted elsewhere. Remove it. Let's be
specific with our sources.list files.

Change-Id: Iefbd59ad20e9fdc450d9a0c4e58b9cf4a89ff5a3
2018-08-17 11:53:52 -05:00
Monty Taylor
03d09b777e
Use openstack and groups inventory from system-config
Rather than copying these out of system-config inside of
install-ansible, just point the ansible.cfg to them in the system-config
location. This way as changes come in that have group updates we don't
have to first apply them to the system.

Change-Id: I1cefd7848b7f3f1adc8fbfa080eb9831124a297b
2018-08-17 11:47:13 -05:00
Monty Taylor
92c9a7c869
Clean up puppet variables and playbooks
The puppet playbooks were some of the first we wrote, so they're
slightly wonky.

Remove '---' lines that are completely unnecessary.

Fix indentation.

Move some variables that are the same everywhere into
ansible variables.

Put puppet related variables into the puppet group_vars.

Stop running puppet on localhost in the git playbook.

Change-Id: I2d2a4acccd3523f1931ebec5977771d5a310a0c7
2018-08-17 09:41:12 -05:00
Monty Taylor
1a8c2f66da
Move /opt/system-config/production to /opt/system-config
The production directory is a relic from the puppet environment concept,
which we do not use. Remove it.

The puppet apply tests run puppet locally, where the production
environment is still needed, so don't update the paths in the
tools/prep-apply.sh.

Depends-On: https://review.openstack.org/592946
Change-Id: I82572cc616e3c994eab38b0de8c3c72cb5ec5413
2018-08-17 09:41:02 -05:00
Monty Taylor
7a0ac4ce03
Set mgmt_puppet_module_dir publically
This was set in the private variables on brige for the transition. But
it can go here now.

Change-Id: I3883672bf549681f8a4f26871c485a71de8ee056
2018-08-17 09:38:35 -05:00
Monty Taylor
c4ae268492
Turn on fact caching
We do this for zuul jobs already, so let's do it for our production
runs.

Shift the inventory cache location down a directory so that launch-node
can invalidate the inventory cache.

Change-Id: I52b1c48d091c07e4205c1a7233448925ca26d8d3
2018-08-17 05:44:37 -05:00
Zuul
f3036203c3 Merge "Remove base.yaml things from openstack_project::server" 2018-08-17 10:43:53 +00:00
Zuul
baa717a1f9 Merge "Add raw blocks to storyboard exim routers" 2018-08-16 22:35:59 +00:00
Monty Taylor
bab6fcad3c
Remove base.yaml things from openstack_project::server
Now that we've got base server stuff rewritten in ansible, remove the
old puppet versions.

Depends-On: https://review.openstack.org/588326
Change-Id: I5c82fe6fd25b9ddaa77747db377ffa7e8bf23c7b
2018-08-16 17:25:10 -05:00
Zuul
ab84adf4bb Merge "Template all exim routers" 2018-08-16 22:19:35 +00:00
Monty Taylor
4b7252ce8b
Add raw blocks to storyboard exim routers
The exim config chunk has a {{ in it, which makes the ansible jinja
very cranky. Add in a raw block so it doesn't try to understand the
exim.

Change-Id: If49d976e503b6ebe236a2d2c6077cce96783e102
2018-08-16 16:29:02 -05:00
Zuul
1463a78fc0 Merge "Make a firehose group with firehose01 in it" 2018-08-16 21:20:46 +00:00
James E. Blair
40c6e6d7ad Template all exim routers
So that we can have complete control of the router order, always
template the full set of routers, including the "default" ones.
So that it's easy to use the defaults but put them in a different
order, define each router in its own variable which can be used
in host or group vars to "copy" that router in.

Apply this change to lists, firehose, and storyboard, all of which
have custom exim routers.  Note that firehose intentionally has
its localuser router last.

Change-Id: I737942b8c15f7020b54e350db885e968a93f806a
2018-08-16 13:49:55 -07:00
Monty Taylor
f78f871afe
Make a firehose group with firehose01 in it
We want to configure firehose logically as the firehose service, but the
host that is in the group is called firehose01.openstack.org. Make a
group and put the config variables for firehose into it.

Change-Id: I17c8e8a72f41c5e2730af81f70cef81dd3ed7bca
2018-08-16 15:11:20 -05:00
Zuul
40fec4c885 Merge "Use jinja match filter instead of regex_match" 2018-08-16 20:10:02 +00:00
Zuul
6f1ad6a6e2 Merge "Use ansible group vars for futureparser flag" 2018-08-16 19:57:05 +00:00
Monty Taylor
3e139891be
Use jinja match filter instead of regex_match
regex_match seems to either not work or not exist or something. match,
otoh, works. Additionally, we get this:

  [DEPRECATION WARNING]: Using tests as filters is deprecated. Instead
   of using `result|match` use `result is match`. This feature will
   be removed in version 2.9.

when using the | syntax, so obey the warning and switch to is.

Change-Id: Ie201241a11c08b9fed58c0e1790e8187ee4cf474
2018-08-16 14:26:17 -05:00
Monty Taylor
b8f4081c2e
Use ansible group vars for futureparser flag
Now that we're running with ansible, we can set the futureparser varible
in the group_vars for the futureparser group and stop passing it as a
parameter explicitly.

Change-Id: I41fe283e96bb48a17f2acfe2ffd939223b5345e7
2018-08-16 14:02:50 -05:00