38 Commits

Author SHA1 Message Date
Ian Wienand
7810230408 Add OVH GRA1 mirror
This mirror will be manually configured with kafs (see
https://review.opendev.org/623974).  This should be a nice distant
geographic counterpoint to the IAD RAX server.

This will need to be manually configured with a custom kernel for now,
but fixes are making their way upstream and this host will be
converted when available.

Depends-On: https://review.opendev.org/667529
Change-Id: I6a22933029c096c781c93c33e6edf03bf59223c9
2019-06-27 10:07:44 +10:00
Ian Wienand
0041f4f673 Add certificates for IAD/ORD opendev.org mirrors
Change-Id: I509517c7601989cff18e29277f3391a813d5ba99
2019-06-12 11:25:50 +10:00
James E. Blair
3199e3b225 Enable SPF checking on lists
This requires an external program and only works on Debian hosts.

Newer versions of exim (4.91) have SPF functionality built-in, but
they are not yet available to us.

Change-Id: Idfe6bfa5a404b61c8761aa1bfa2212e4b4e32be9
2019-06-07 10:34:33 -07:00
Ian Wienand
2e9992af9e mirror01.dfw.rax.opendev.org : use python3 for ansible
This is a bionic host, so requires this to run as it has no
/usr/bin/python.  This is the same as the other bionic hosts, I just
forgot it.

Change-Id: Ifdd1df2fa83dd25dcc20596ce17e2f0c88279c62
2019-05-22 10:03:11 +10:00
Ian Wienand
73bbc6787f Bringup mirror01.dfw.rax.opendev.org
This is an initial host for testing opendev.org mirrors

Change-Id: I26b9ed1e21e2111f48bc7ecc384880c274eed213
Depends-On: https://review.opendev.org/660235
2019-05-21 11:08:30 +10:00
Ian Wienand
733122f0df Use handlers for letsencrypt cert updates
This change proposes calling a handler each time a certificate is
created/updated.  The handler name is based on the name of the
certificate given in the letsencrypt_certs variable, as described in
the role documentation.

Because Ansible considers calling a handler with no listeners an error
this means each letsencrypt user will need to provide a handler.

One simple option illustrated here is just to produce a stamp file.
This can facilitate cross-playbook and even cross-orchestration-tool
communication.  For example, puppet or other ansible playbooks can
detect this stamp file and schedule their reloads, etc. then remove
the stamp file.  It is conceivable more complex listeners could be
setup via other roles, etc. should the need arise.

A test is added to make sure the stamp file is created for the
letsencrypt test hosts, which are always generating a new certificate
in the gate test.

Change-Id: I4e0609c4751643d6e0c8d9eaa38f184e0ce5452e
2019-05-14 08:14:51 +10:00
Jeremy Stanley
74649101f8 Blackhole spam for airship-discuss-owner address
The airship-discuss-owner address for lists.airshipit.org is now
besieged with a flood of unsolicited messages. Reject anything sent
to it with an SMTP error explaining the situation.

Change-Id: I19fcea2a502c41cc9438f2710dae3cd686eecc05
2019-04-23 18:12:11 +00:00
Ian Wienand
a88d836661 Enable production cert generation for graphite01
Testing with the staging cert has shown nothing is going crazy and
making unreasonable letsencrypt requests ... switch this to generate a
real cert.

Change-Id: I861ea295312f83c66dd9b37271969d6e7f8fc2a2
2019-04-11 10:32:30 +10:00
Ian Wienand
86c5bc2b45 letsencrypt: split staging and self-signed generation
We currently only have letsencrypt_test_only as a single flag that
sets tests to use the letsencrypt staging environment and also
generates a self-signed certificate.

However, for initial testing we actually want to fully generate
certificates on hosts, but using the staging environment (i.e. *not*
generate self-signed certs).  Thus we need to split this option into
two, so the gate tests still use staging+self-signed, but in-progress
production hosts can just using the staging flag.

These variables are split, and graphite01.opendev.org is made to
create staging certificates.

Also remove some debugging that is no longer necessary.

Change-Id: I08959ba904f821c9408d8f363542502cd76a30a4
2019-04-10 08:47:32 +10:00
Ian Wienand
00efd089a2 Really add graphite01.opendev.org certificate details
This file was accidentally dropped from
I3e762d071cc609856950898b36f1903fe52840a6 during a rebase.

Change-Id: Iabc1db2aa029d7ff73b742ed63d367d8daa39187
2019-04-09 12:06:23 +10:00
Clark Boylan
9342c2aa6d Add zuul user to bridge.openstack.org
We want to trigger ansible runs on bridge.o.o from zuul jobs. First
iteration of this tried to login as root but this is not allowed by our
ssh config. That config seems reasonable so we add a zuul user instead
which we can ssh in as then run things as root from zuul jobs. This
makes use of our existing user management system.

Change-Id: I257ebb6ffbade4eb645a08d3602a7024069e60b3
2019-03-04 14:47:51 -08:00
James E. Blair
175a337e01 Handle registry role running under py3
Also, correct the host_vars filename.  Again.
Also, make sure we run the test on changes to the host_vars filename.

Change-Id: I95fb61531bae677f5c68f4e56ed718da6c507eb9
2019-02-08 09:13:06 -08:00
James E. Blair
ab557b2a98 Correct insecure-ci-registry01 hostvar filename
It's in opendev, not openstack.

Change-Id: Ib2f4154b18e6c15210e1f6ded9c1628b2fe5ef7a
2019-02-07 13:32:35 -08:00
James E. Blair
2766ec2595 Tell ansible to use py3 on insecure-ci-registry01
Change-Id: If0cf34f55524b7c87133fe04abb5513ad57e893b
2019-02-07 09:44:39 -08:00
James E. Blair
7610682b6f Configure .kube/config on bridge
Add the gitea k8s cluster to root's .kube/config file on bridge.

The default context does not exist in order to force us to explicitly
specify a context for all commands (so that we do not inadvertently
deploy something on the wrong k8s cluster).

Change-Id: I53368c76e6f5b3ab45b1982e9a977f9ce9f08581
2019-02-06 15:43:19 -08:00
Zuul
133116a430 Merge "Save a copy of all messages to openstack-discuss" 2019-02-05 21:07:47 +00:00
Jeremy Stanley
fea54bc43c Move OpenStackID host_vars to group_vars
Both staging and production OpenStackID servers are being updated to
our enumerated host naming convention as part of their upgrade from
Ubuntu Trusty to Xenial. Move their host-specific Ansible variables
to the new host groups we've created for each of them.

Change-Id: I359a51812b749bf9937943bae1cf1850bc1f85c3
2019-02-05 17:19:17 +00:00
James E. Blair
9e6db28ffb Save a copy of all messages to openstack-discuss
To debug DMARC issues, save a copy of every message sent to
openstack-discuss with as little manipulation as possible.

Change-Id: Ic1156849957bc326e9216c2aca0ab9d180e158e6
2019-01-23 11:19:00 -08:00
Jeremy Stanley
159f012d47 Reject messages to starlingx-discuss-owner
The owner address for the starlingx-discuss list on
lists.starlingx.io has started receiving large volumes of
unsolicited messages unrelated to its intended purpose. As there's
no easy way to discern them from legitimate messages, we'll do the
same as we've done for other owner addresses and reject them with a
brief error explaining the situation.

Change-Id: I95a910c2e6206098ca268a0e10e86b66455ad1bd
2019-01-02 19:40:15 +00:00
Jeremy Stanley
a507b6b401 Add lists.opendev.org to Mailman
Set up the initial boilerplate to enable addition of new
project-neutral Mailman mailing lists on lists.opendev.org.

Change-Id: I8cad4149bdd7b51d10f43b928cdb9362d4bde835
2018-12-13 20:36:08 +00:00
Jeremy Stanley
8017415779 Retire the interop-wg mailing list
This list's owners have asked for it to be shut down, as they will
be using an [interop-wg] tag on the new openstack-discuss ML for
future communication. Once this merges (so that Puppet won't
recreate it), the list can be removed with the `rmlist` utility
(this will still leave the archives available but will remove it
from the list index and no longer accept subscriptions/posts).

Set the old list address as an alias for the new openstack-discuss
ML so that replies to previous messages from the list will be routed
there for the foreseeable future.

Change-Id: Ib5fd5aece2465d569e0e7c180ee14ba94882f2b7
2018-11-30 18:39:16 +00:00
Jeremy Stanley
e9d49b4839 Shut down openstack general, dev, ops and sigs mls
The general openstack, openstack-dev, openstack-operators and
openstack-sigs mailing lists have been deprecated since November 19
and are slated to be removed on December 3. Merging this on that
date will ensure any further replies to messages from those lists
are rerouted to the new openstack-discuss mailing list for the
foreseeable future.

The openstack-tc list is included in this batch as it has already
been closed down with a recommendation to send further such
communications to the openstack-discuss ML.

Additionally remove the Puppet mailman resource for the
openstack-sigs ML so it won't be automatically recreated after it
gets deleted (the other lists predate our use of Puppet for this
purpose).

Clean up the corresponding -owner spam rejection aliases since these
addresses will no longer be accepting E-mail anyway.

Change-Id: I9a7fae465c3f6bdcf3ebbadb8926eb4feb8fad79
2018-11-30 18:22:00 +00:00
Ian Y. Choi
72781811aa Blackhole messages to openstack-ko-owner@l.o.o
The OpenStack Korean mailing list's owner address have
become overrun by the same mass spam we've seen hitting our other ML
owner addresses. Add a blackhole alias for it.

Change-Id: Ia6c7e6701a69ee56076062aa85f8699121648501
2018-11-29 02:23:35 +09:00
Jeremy Stanley
33ec337b42 Blackhole messages to openstack-sigs-owner@l.o.o
The OpenStack SIGS mailing list's owner address is starting to
become overrun by the same mass spam we've seen hitting our other ML
owner addresses. Add a blackhole alias for it.

Change-Id: Iefc5b5fa600c5d1de75d3302c8ddf0e1a03301e5
2018-11-19 16:16:33 +00:00
Jeremy Stanley
4fb2143f3c Blackhole messages to edge-computing-owner@l.o.o
The OpenStack edge-computing mailing list's owner address is
starting to become overrun by the same mass spam we've seen hitting
our other ML owner addresses. Add a blackhole alias for it.

Change-Id: I97a2db5d0565cc166604352e397f580ea2d9e767
2018-11-12 10:15:02 +00:00
James E. Blair
2780973330 Set ansible python version for opendev nameservers
Change-Id: Icaee291e872b6a19793a1ba003e55a43f3898ea7
2018-11-01 09:20:26 -07:00
James E. Blair
46a643f823 lists: run mailman verp router before dnslookup
The mailman verp router handles remote addresses like dnslookup.
It needs to run before dnslookup in order to be effective, so run
it first.  It's only for outgoing messages, not incoming, so won't
affect the blackhole aliases we have for incoming fake bounce
messages.

Note that the verp router hasn't been used in about a year due to
this oversight, so we should merge this change with caution.

Change-Id: I7d2a0f05f82485a54c1e7048f09b4edf6e0f0612
2018-10-16 13:04:13 -07:00
James E. Blair
c49d5d6f2b Allow Zuul to log into bridge
Allow post-review jobs running under system-config and project-config
to ssh into bridge in order to run Ansible.

Change-Id: I841f87425349722ee69e2f4265b99b5ee0b5a2c8
2018-09-12 10:20:26 -06:00
Monty Taylor
a634593a05
Set mgmt_hieradata in puppet group_vars
This is not a variable describing the system-under-management
bridge.openstack.org - it's a variable that is always true for all
systems in the puppet group.

As a result, update the puppet apply test to figure out which directory
we should be copying modules _from_ - since the puppet4 tests will be
unhappy otherwise.

Change-Id: Iddee83944bd85f69acf4fcfde83dc70304386baf
2018-08-17 14:25:50 -05:00
Monty Taylor
7a0ac4ce03
Set mgmt_puppet_module_dir publically
This was set in the private variables on brige for the transition. But
it can go here now.

Change-Id: I3883672bf549681f8a4f26871c485a71de8ee056
2018-08-17 09:38:35 -05:00
Zuul
f3036203c3 Merge "Remove base.yaml things from openstack_project::server" 2018-08-17 10:43:53 +00:00
Monty Taylor
bab6fcad3c
Remove base.yaml things from openstack_project::server
Now that we've got base server stuff rewritten in ansible, remove the
old puppet versions.

Depends-On: https://review.openstack.org/588326
Change-Id: I5c82fe6fd25b9ddaa77747db377ffa7e8bf23c7b
2018-08-16 17:25:10 -05:00
James E. Blair
40c6e6d7ad Template all exim routers
So that we can have complete control of the router order, always
template the full set of routers, including the "default" ones.
So that it's easy to use the defaults but put them in a different
order, define each router in its own variable which can be used
in host or group vars to "copy" that router in.

Apply this change to lists, firehose, and storyboard, all of which
have custom exim routers.  Note that firehose intentionally has
its localuser router last.

Change-Id: I737942b8c15f7020b54e350db885e968a93f806a
2018-08-16 13:49:55 -07:00
Monty Taylor
f78f871afe
Make a firehose group with firehose01 in it
We want to configure firehose logically as the firehose service, but the
host that is in the group is called firehose01.openstack.org. Make a
group and put the config variables for firehose into it.

Change-Id: I17c8e8a72f41c5e2730af81f70cef81dd3ed7bca
2018-08-16 15:11:20 -05:00
Monty Taylor
0d1f235fce
Add exim config for firehose and storyboard
In order to get puppet out of the business of mucking with exim and
fighting ansible, finish moving the config to ansible.

This introduces a storyboard group that we can use to apply the exim
config across both servers. It also splits the base playbook so that we
can avoid running exim on the backup servers. And we set
purge_apt_sources the same as was set in puppet. We should probably
remove it though, since none of us have any clue why it's here.

Change-Id: I43ee891a9c1beead7f97808208829b01a0a7ced6
2018-08-15 15:11:48 -05:00
Monty Taylor
4cca3f8d2a
Add lists exim config to ansible
The mailing list servers have a more complex exim config. Put the
routers and transports into ansible variables.

While we're doing it, role variables with an exim_ prefix - since 'routers'
as a global variable might be a little broad.

iteritems isn't a thing in python3, only items.

We need to escape the exim config with ${if or{{ - because of the {{
which looks like jinja. Wrap it in a {% raw %} block.

Getting the yaml indentation right for things here is non-trivial. Make
them strings instead.

Add a README.rst file - and use the zuul:rolevar construct in it,
because it's nice.

Change-Id: Ieccfce99a1d278440c5baa207479a1887898298e
2018-08-15 15:11:48 -05:00
Monty Taylor
ee02ba0123
Set mgmt_hieradata variable for bridge.openstack.org
ansible-role-puppet attempts to infer where it should copy hieradata
from based on puppet3 or puppet4. On bridge there is no puppet and thus
there is no puppet version. Set mgmt_hieradata to tell
ansible-role-puppet from where it should copy hiera secrets.

Change-Id: I0c518b8a5a8ee2155e2125d6bc7f4e0a3bf4faeb
2018-08-10 12:21:39 -05:00
Monty Taylor
60fecd508d
Install and configure ansible on bridge
There is a shared caching infrastructure in ansible now for inventory
and fact plugins. It needs to be configured so that our inventory access
isn't slow as dirt.

Unfortunately the copy of openstack.py in 2.6 is busted WRT to caching
because the internal API changed ... and we didn't have any test jobs
set up for it. This also includes a fixed copy of the plugin and
installs it into the a plugin dir.

Change-Id: Ie92e5d7eac4b7e4060a4e07cb29c5a6f2a16ae18
2018-08-03 09:05:07 -05:00