465 Commits

Author SHA1 Message Date
Monty Taylor
655ae66c84 Pin mariadb to 10.4
Upgrading across versions like 10.3 to 10.4 apparently takes some
care and shouldn't be done accidentally.

Change-Id: I42e6cbd780dca965d2e0e33ba594cdb5fab21d4b
2019-03-05 08:18:44 +00:00
Zuul
f1f2e3f843 Merge "Run docker-compose pull before docker-compose up" 2019-03-04 23:59:26 +00:00
Clark Boylan
9342c2aa6d Add zuul user to bridge.openstack.org
We want to trigger ansible runs on bridge.o.o from zuul jobs. First
iteration of this tried to login as root but this is not allowed by our
ssh config. That config seems reasonable so we add a zuul user instead
which we can ssh in as then run things as root from zuul jobs. This
makes use of our existing user management system.

Change-Id: I257ebb6ffbade4eb645a08d3602a7024069e60b3
2019-03-04 14:47:51 -08:00
James E. Blair
4d91f29b39 Run docker-compose pull before docker-compose up
This will make sure that the latest relevant images are in the
local image storage, and therefore, will cause docker-compose up
to recreate containers when the images are updated.

Change-Id: Ic6f0bc8c8aea5b5c16501f4ab5d3095fb70c0ff7
2019-03-04 14:40:35 -08:00
Monty Taylor
e29a62d87f Filter setup-repos loop before include_tasks
Ansible task execution can be a bit slow - so the noop case where
we don't end up doing anything can still be costly. Instead, put
the when on the loop call, which will apply to each iteration of
the loop, not running the loop itself. This way we should only
include_tasks if we need to.

In order for the utility playbook to keep working, we also run all
of the iterations of the loop if gitea_always_update is true. This
will make a sync run take a long time but be comprehensive.

Change-Id: Ib60c736d46d8253e603de097eb80bc84b3366310
2019-03-04 20:39:17 +00:00
Monty Taylor
d3220a7ade Add utility playbook for fixing gitea project settings
We normally only do project settings when we create the project.
Add a playbook we can use to do a manual sync from project-config.

Change-Id: I5260f2de697420a01d796acc3128be00705a53ee
2019-03-04 16:06:49 +00:00
Monty Taylor
1b7d793f7d Only update gitea project settings during creation
It takes too long to update them all every time. Only update them
when we create the project.

Change-Id: I2a2b2515b57a4e6a494a7c73dd86e562816a31ea
2019-03-04 16:03:57 +00:00
Monty Taylor
626bc0459a Limit project description to 255 characters
Otherwise, gitea is sad.

Change-Id: I36e466d06daab8147c052ba3b8fe6754799205b5
2019-03-04 15:57:09 +00:00
Zuul
fe89631029 Merge "Add gitea to project rename playbook" 2019-03-04 15:37:55 +00:00
Monty Taylor
b076aefa08 Revert "Set default branch for repo"
This is 404ing. It's not strictly needed for the moment, let's
come back to it.

This reverts commit 266b2dd3fc4bcbe2416f77f34540705a6ba6cdde.

Change-Id: Id9ab3c233753025d7a01eb4664ecee6d102bbf19
2019-03-03 21:55:11 +00:00
Monty Taylor
ef42937daf Parse the CRSF token before using
We're missing a step here.

Change-Id: I01ea5e1570afe8f59e96953dc7dc30372a490bf0
2019-03-03 17:57:31 +00:00
Monty Taylor
266b2dd3fc Set default branch for repo
If, for some reason, a repo gets created via gerrit replication
instead of via the api, the default branch setting can be incorrectly
inferred. While we're setting things, just set master to be the default
branch everywhere.

Change-Id: I388afe670221bd8ec28fbba71041ab460e66411b
2019-03-03 16:31:19 +00:00
Monty Taylor
72c3904717 Use HTTP for repo settings
This uses the form post approach also used in repo renaming. There
is no official REST API for this yet, so the form post values were
taken from what the web ui does.

We should submit a patch upstream to get an actual api call for this.

Run it idempotently, submitting the settings for every repo every time.

Change-Id: I9265837039df962e85f11d16419e043fb9a56ff8
2019-03-03 15:47:15 +00:00
Zuul
abee79164b Merge "Add edit-secrets script to bridge.o.o" 2019-03-02 21:11:23 +00:00
Clark Boylan
0fe4c04172 Aggregate all gitea repos and check against them
We are currently attempting to create repos which already exist. This
fails.

The reason for this is we set the gitea_org_repos fact for each org
overriding the last org. This means only the last org processed has any
projects in this list. We then check against this list when creating
projects so that we only create projects if they aren't in the list.
Meaning any project for repos not in the last org attempts to get
recreated.

We can address this by keeping a global list of repos regardless of org
then checking against that.

An alternative solution would be to process projects for each org
separately. Or to have gitea give us the global list so we don't have to
build it ourselves.

Change-Id: Id9a480634918dad2160a4e040a41ce6226ae67d8
2019-03-01 20:49:30 -08:00
Clark Boylan
c0b64767a2 Use heredoc in gitea sql_statement
Switch to a heredoc for the gitea sql_statement so that we don't have to
figure out multiple levels of "" quoting.

Change-Id: I734640936a9d15d03026fc3b05b5fbc221957b94
2019-03-01 17:54:02 -08:00
Monty Taylor
619d3e42b1 Update playbook doc comment for correctness
Change-Id: If5260a9e33aadc912f95960f287740dc3f10b7a2
2019-03-01 22:16:44 +00:00
Monty Taylor
e836fb7af6 Remember that lookups are local
Go ahead and actually clone project-config on bridge, because
we're going to read the projects.yaml file with a lookup. Because
it's a local action, not a remote action.

Change-Id: I77454bcb10b797ce5b48018caef7fecb31947b97
2019-03-01 21:55:18 +00:00
Zuul
43a7014c88 Merge "Clone project-config to /opt on gitea servers" 2019-03-01 20:16:59 +00:00
Zuul
cd9f48431a Merge "Create git repos on gitea servers before gerrit" 2019-03-01 20:16:53 +00:00
Zuul
683056137c Merge "Update nsd systemd unit deps" 2019-03-01 18:51:18 +00:00
James E. Blair
2a6de1e8bf Clone project-config to /opt on gitea servers
We don't need to clone to /tmp, /opt is safer.

Change-Id: Ib41e5e22f6de9048db12f9814dc4af4621e1637a
2019-03-01 10:07:59 -08:00
Monty Taylor
a53f333162 Create git repos on gitea servers before gerrit
We need to make sure repos are created in gitea before gerrit so
that we can safely create to gerrit and have it replicate.

Change-Id: If3efc7ed0d7995f1ef6f52cd2eefb9260193e020
Story: 2004627
Task: 29703
2019-03-01 18:07:33 +00:00
James E. Blair
7da7002dc1 Add gitea to project rename playbook
Change-Id: Idca623ff59f3318c762ffd5c65e3c739fac819af
Story: 2004627
Task: 29706
2019-03-01 09:30:46 -08:00
James E. Blair
36323c274e Open port 80 on zuul-preview
Change-Id: I48525690253bf31ad35bdbcbbeccc12c9de0d0be
2019-03-01 08:21:00 -08:00
Monty Taylor
04077adf29 Set ansible_python_interpreter for zuul-preview
Change-Id: I17cde074fee4444d387b070acfdd4ad150bc1f4e
2019-02-28 23:07:54 +00:00
Monty Taylor
8e52ae742d Add gitea servers to inventory and load-balancer hostvars
We have new servers, and a new load balancer server, let's use them.

Change-Id: I5c58530b66637759c7882abb96967d0733bc8e6d
2019-02-28 17:13:02 +00:00
Clark Boylan
3ec0861e6b Add edit-secrets script to bridge.o.o
This adds a script that will wrap emacs with gpg-agent when editing the
secrets file. This avoids issues with rogue gpg-agents running on the
system.

Change-Id: Ic3cc73b5c25eab2ede41d8ca05b5695b817973d9
2019-02-27 08:45:11 -08:00
Clark Boylan
52a4bae170 Apply package updates before we reboot in launch-node
When launching a new server we should make sure that all available
package updates are installed before we reboot the server. This way we
get available security updates applied to things like our kernel.

This change adds a new playbook that runs the unattended-upgrade command
on debuntu servers. Will need to add support for other platforms in a
followup change.

Change-Id: Idc88dc33afdd209c388452493e6a7f5731fa0974
2019-02-26 14:17:23 -08:00
James E. Blair
287eecd9d2 Run zuul-preview
Change-Id: Ib72e2bd29d1061822e0c16c201445115a5e5c58f
2019-02-25 13:14:51 -08:00
Zuul
d96623934c Merge "Run an haproxy load balancer for gitea" 2019-02-22 23:00:11 +00:00
Zuul
0567b59bec Merge "Use host networking for gitea" 2019-02-22 21:42:43 +00:00
James E. Blair
4b031f9f24 Run an haproxy load balancer for gitea
This runs an haproxy which is strikingly similar to the one we
currently run for git.openstack.org, but it is run in a docker
container.

Change-Id: I647ae8c02eb2cd4f3db2b203d61a181f7eb632d2
2019-02-22 12:54:04 -08:00
Zuul
04cc048993 Merge "run-base: configure docker mirrors on all hosts in CI" 2019-02-22 20:31:27 +00:00
Zuul
1e71c37df7 Merge "Don't install a blank docker daemon config" 2019-02-22 20:26:40 +00:00
James E. Blair
dd011e1c7a run-base: configure docker mirrors on all hosts in CI
When setting up hosts for testing in CI, configure the docker
mirrors before running the base playbook.

Change-Id: I172ae87156238fa6a07414c74e1ca17df1a30257
2019-02-22 08:42:12 -08:00
James E. Blair
480c7ebe37 Use host networking for gitea
Change-Id: If706c6f85022919add93e46eeb6eae1b6d948d75
2019-02-21 15:27:44 -08:00
Zuul
0c59702709 Merge "Turn on future parser for afs servers" 2019-02-21 21:55:06 +00:00
Jeremy Stanley
371d9e8a1c Add site name to Exim local_domains on SB server
To allow Exim to perform outbound delivery for the production
StoryBoard service without choking on sender verification, add
storyboard.openstack.org to its local_domains list.

Change-Id: I2d380ead26319269819026acb2e1b175774a6c77
2019-02-21 16:00:41 +00:00
James E. Blair
bf2d53eb7d Don't install a blank docker daemon config
It's not necessary to install an empty config file, and doing so
will prevent us from using other roles to configure mirrors on
test hosts.

Change-Id: I3d7eb615f1e297fde2d693b5fc64bc6e691e2c22
2019-02-20 09:09:52 -08:00
Ian Wienand
423c90e72d Allow nb*.openstack.org to talk to graphite
The builders actually produce a range of interesting stats on the
images they build; allow them to actually send them.

Change-Id: If40df406d505a4d37a6fc72b44d0acb5c6d7e7dd
2019-02-20 08:06:21 +11:00
Zuul
7a39eaafcf Merge "Deploy gitea with docker-compose" 2019-02-19 20:27:15 +00:00
James E. Blair
67cda2c7df Deploy gitea with docker-compose
This deploys a shared-nothing gitea server using docker-compose.
It includes a mariadb server.

Change-Id: I58aff016c7108c69dfc5f2ebd46667c4117ba5da
2019-02-18 08:46:40 -08:00
James E. Blair
eeb69e7ba2 Fix htpasswd creation for registry
We were using the wrong filename and encryption type.

Change-Id: I48d2a7288e5cd52e57f5632fee17f677429a1a9c
2019-02-16 11:59:56 -08:00
James E. Blair
e9e1535646 Temporarily stop installing openshift
Change-Id: If7a1070da62fb1d1aec5c5c530e0af174e3c3cf8
2019-02-14 10:19:59 -08:00
Zuul
d3e554e306 Merge "Stop running k8s-on-openstack nested" 2019-02-12 22:22:06 +00:00
Monty Taylor
0c4a981f73 Stop running k8s-on-openstack nested
The current code runs k8s-on-openstack's ansible in an ansible
task. This makes debugging failures especially difficult.

Instead, move the prep task to update-system-config, which will
ensure the repo is cloned, and move the post task to its own
playbook. The cinder storage class k8s action can be removed from
this completely as it's handled in the rook playbook.

Then just run the k8s-on-openstack playbook as usual, but without
the cd first so that our normal ansible.cfg works.

Change-Id: I6015e58daa940914d46602a2cb64ecac5d59fa2e
2019-02-12 18:17:46 +00:00
Zuul
c820963613 Merge "Install kubectl on bridge" 2019-02-11 22:02:36 +00:00
James E. Blair
94d404a535 Install kubectl on bridge
With a snap package.  Because apparently that's how that's done.

Change-Id: I0462cc062c2706509215158bca99e7a2ad58675a
2019-02-11 10:16:58 -08:00
James E. Blair
0d0e704267 Open port 5000 on the docker intermediate registry
Change-Id: I444356c3b3f0b8d5796b80ff125d381d03e2aaba
2019-02-11 09:57:14 -08:00