54 Commits

Author SHA1 Message Date
Andrii Ostapenko
67acec14cd Refactor to support jobs invocations from other projects
Switch osh fuctional jobs to osh-infra-job-runner playbook inheriting
from openstack-helm-infra-functional.
Specify gate_scripts_relative_path var for each job to support the
case when corresponding jobs defined in osh are invoked from another
project to allow playbook know where to look for gate scripts.

Depends-On: https://review.opendev.org/740557
Change-Id: I315f04ef7bdf5bf97d09843de8f878e2d15377dc
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-07-12 03:19:39 +00:00
Tin Lam
918a307427 feat(tls): add tls support to openstack services
This patch set enables TLS for the following OpenStack services: keystone,
horizon, glance, cinder, heat, nova, placement and neutron for s- (stein)
and t- (train) release. This serves as a consolidation and clean up patch
for the following patches:

[0] https://review.opendev.org/#/c/733291
[1] https://review.opendev.org/#/c/735202
[2] https://review.opendev.org/#/c/733962
[3] https://review.opendev.org/#/c/733404
[4] https://review.opendev.org/#/c/734896

This also addresses comments mentioned in previous patches.

Co-authored-by: Gage Hugo <gagehugo@gmail.com>
Co-authored-by: sgupta <sg774j@att.com>

Depends-on: https://review.opendev.org/#/c/737194/

Change-Id: Id34ace54298660b4b151522916e929a29f5731be
Signed-off-by: Tin Lam <tin@irrational.io>
2020-07-10 09:36:31 -05:00
Gage Hugo
6027ac0c0c Cleanup zuul checks
This change moves the tungsten fabric check down to the periodic
checks with the other feature overrides, removes several duplicate
checks that are currently non-functional, and moves the rest to
experimental.

Also moves the compute-kit jobs back to using the default
single-node nodeset due to cloudedge going down and not
enough available 32gb nodes as a result

Change-Id: Ib4f314d5246af8756611346ed4919492d98510a5
2020-07-09 18:38:56 +00:00
Zuul
135e2c8b64 Merge "Add TungstenFabric compatibility to charts" 2020-07-08 13:47:49 +00:00
Chinasubbareddy Mallavarapu
a385c18176 [CEPH] OSH: use loopback devices for ceph osds.
- This is to make use of loopback devices for ceph osds since
support for directory backed osds going to depricate.

- Move to bluestore from filestore for ceph-osds.

Change-Id: Ia95c9ceb81f7d253dd582a2e753a6ed8fe60a04d
2020-06-30 17:05:22 -05:00
OlegBravo
4f2eb8cac6 Add TungstenFabric compatibility to charts
The charts changes are required for deployment
of various clouds based on Tungsten Fabric SDN.
Right now it's tested for Airship-in-a-bottle.

The code cannot be tested currently in
OpenStack Helm project because of absence of
tests and platform for that.

This patchset doesn't have Heat-related changes,
they'll be added later.

Change-Id: I73f2ced2b09dbb93146334b59fe4571fa13dbfb0
Depends-On: https://review.opendev.org/#/c/734635/
2020-06-26 10:39:20 +03:00
Andrii Ostapenko
81884d8a60
Switch compute-kit jobs to 32GB nodes
We're running compute kit jobs on the edge of current flavor
capabilities and actually exceeding it with TLS enabled.
With this change compute kit TLS commit successfully pass check
jobs.

Change-Id: Ide1a1600f2e19fcb91ec7d90c8f316283b2d9697
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-06-21 18:14:59 -05:00
Tin Lam
26c795aa0d feat(tls): adding cert manager to gate
Change-Id: I913c9de3bd405ecd4482c6d455c9c7399e64f95a
Signed-off-by: Tin Lam <tin@irrational.io>
2020-05-20 19:59:58 +00:00
diwakar thyagaraj
a61050c2b3 Enable Apparmor to init containers for Heat Components
Also added Ingress apparmor Fix.

Change-Id: I6f4a1e6778b16c855072c0d6583e61af86f252a6
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-05-07 08:30:28 -05:00
Tin Lam
4675f69d39 Cleanup OSH infra jobs
This patch set cleans up and moves some jobs to experimental and
periodic to reduce the resources need on infra.

Change-Id: I7567b57521a36e7da6490c4e65522b32eccf5a51
Signed-off-by: Tin Lam <tin@irrational.io>
2020-04-03 09:10:50 -05:00
dt241s@att.com
1c46490094 Enable Apparmor to Cinder
Co-Authored by gage Hugo <gagehugo@gmail.com>.

This Patchset creates Zuul Gate Jobs for apparmor to support Cinder.

Change-Id: I7705512a3b50560b183e19f0868be40078241cdd
2020-04-01 04:27:56 +00:00
Gage Hugo
a23f0c4d5b Only run horizon check for horizon changes
This change sets the files attribute for the zuul job that runs
the horizon check to only run that check when a file change is made
under the horizon chart directory.

Change-Id: I3703b39216d6139f28d00b7fc0bf098514bdbe6d
2020-03-28 23:04:38 +00:00
rajesh.kudaka
78d1624ad3 Add netpol value overrides for placement chart
Change-Id: Iea52d94a1b5cf1db41ce694349e49e6311a47333
2020-03-19 16:53:37 +00:00
Tin Lam
f1bdcc3251 Fix network policy job
This patch set addresses an issue with the placement component of nova
breaking the network policy job.

Also, make the network policy jobs voting to ensure things do not break.

Change-Id: I41dfa6a335a915dbaf08114c2e14e906c76e85ba
Signed-off-by: Tin Lam <tin@irrational.io>
2020-03-16 16:05:06 +00:00
KAVVA, JAGAN MOHAN REDDY (jk330k)
3e3b9436b7 Enable Docker default AppArmor profile to barbican
This adds default Apparmor profile to barbican

Change-Id: Ieb3465fa0caa99fb9b1906c0173e35cd936fc4d4
2020-03-08 15:18:33 +00:00
Zuul
5de7b5e26c Merge "Add train release support" 2020-02-28 22:50:35 +00:00
Tin Lam
2aa32665b4 Add train release support
This patch set adds in job to test the OpenStack train releases.

Depends-On: https://review.opendev.org/#/c/706456/
Change-Id: I89fef1264f68dab7e921a9e5503c29d6a051f342
Signed-off-by: Tin Lam <tin@irrational.io>
2020-02-28 20:19:58 +00:00
dt241s@att.com
0c32a86733 Enable Apparmor to Horizon Components
Change-Id: I7b28eb5e7f571b050ee3ac5eeb2b6868f0d44f58
2020-02-27 18:33:13 +00:00
Gage Hugo
a1fc694ae9 Move apparmor to use feature gates
This change refactors the apparmor job to utilize the feature
gates system instead of relying on separate scripts.

Also disabled barbican running in the apparmor job temporarily
until the correct profile gets used and it can deploy
succesfully.

Change-Id: Iadacd214de3fdb06e4acde4433c5fa86973371d5
2020-01-31 22:24:55 +00:00
Tin Lam
12bee1bb97 Migrate default release to Stein
This patch set updates the default job to use OpenStack Stein release.
The previously default Ocata release will be place in separate job.

Change-Id: I489324f762a179a2cab5499a6d8e57e97c81297f
Signed-off-by: Tin Lam <tin@irrational.io>
2020-01-09 10:00:31 -06:00
Tin Lam
bf434ffd67 Add capability for using FQDN in nova compute
This patch set adds in a capability for the user to defaultly use a
FQDN for the nova compute hostname and the hypervisor hostname when
the host is not explicitly specified in the .Values.conf override.

Change-Id: I3243068dfe91ebb97b3885002296a0f454822ec5
Co-authored-by: Drew Walters <andrew.walters@att.com>
Signed-off-by: Tin Lam <tin@irrational.io>
2020-01-01 20:01:25 +00:00
Steve Wilkerson
f6fe35d452 Disable kubeadm-aio jobs while issues addressed
This disables the keystone-auth single node job and all multinode
periodic and experimental jobs while standing issues with the
kubeadm-aio image deployment are sorted out

Change-Id: I4e1de001ddf17b3c035ca174b7ef8acec8f2bf2c
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-12-18 09:55:50 -06:00
Tin Lam
feb97a2dc3 Add horizon ingress override
This patch set adds in default horizon ingress overrides.

Change-Id: I5a7e8197b84bc5f1ad94d5d6a1d0662257404994
Signed-off-by: Tin Lam <tin@irrational.io>
2019-10-16 02:46:28 +00:00
Huang, Sophie (sh879n)
313fe0ca3e Add keystone ingress netpol
Change-Id: I75874b475039c1f7469f11e02e2231254cc9d8ca
Signed-off-by: Huang, Sophie (sh879n) <sh879n@att.com>
2019-10-08 10:41:38 -05:00
Gage Hugo
c3e085b800 Add network policy nonvoting checks
This change adds two network policy zuul checks, one for the compute-kit,
and one for cinder/ceph, to test network policy for each OpenStack
service. These checks will be non-voting initially.

The network policy rules for each service will initially allow all
traffic. These ingress/egress rules will be defined in future changes
to only explicitly allow traffic between services that are explicitly
allowed to communicate, other traffic will be denied.

Depends-On: https://review.opendev.org/#/c/685130/

Change-Id: Ide2998ebb2af2832f24ca7abc398a82e4a6d70e3
2019-09-26 11:57:15 -05:00
Itxaka
e45621209b
Enable voting for jobs
As agreed on the weekly meeting, its time to enable the jobs
so we dont break them with new changes

Change-Id: I567a19b81a6d780f8a461d62a5d17531f9c1be1b
2019-09-25 09:56:53 +02:00
Steve Wilkerson
f9b1643409 Remove service specific periodic jobs that add no value
This removes the service specific job definitions that currently
run periodically, as they add no value (as they're currently part
of the compute kit jobs).  This helps ensure we have a workable
history of our multinode periodic jobs, as the current number of
periodic jobs limits us to ~2 days of history with no added value

Change-Id: Id525ca4895de2673bed2b638b816834bcf34e131
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-07-11 08:04:28 -05:00
Zuul
ac3952a720 Merge "Add docker-default (enforce) AppArmor profile to barbican" 2019-07-10 09:34:18 +00:00
Pete Birley
74e4474ec6 OpenStack: Check Stein release on Ubuntu Bionic
This PS adds checks for the Stein Release of OpenStack in Ubuntu Bionic
containers.

Depends-On: https://review.opendev.org/667726

Change-Id: Icfad3434ca496a841993b95adaf5d853728d920f
Signed-off-by: Pete Birley <pete@port.direct>
2019-07-03 00:26:30 +00:00
Jagan Kavva
47677e5c4b Add docker-default (enforce) AppArmor profile to barbican
Change-Id: I2011773df00269b59c295afc6de7d24d9704c3ba
2019-06-26 12:04:04 -05:00
Pete Birley
e5f8fcf728 Container Distro: Add checks for OS (rocky) in Ubuntu Bionic
This PS adds checks for running the Rocky release of Openstack under
Python3 in Ubuntu Bionic containers.

Change-Id: I269cef9f8f157e22f6b857822df9a8960dac6ea8
Signed-off-by: Pete Birley <pete@port.direct>
2019-06-26 10:36:26 -05:00
Pete Birley
6606c8bc2e OpenStack: Check Rocky release on Ubuntu Xenial
This PS adds checks for the Rocky Release of OpenStack in Ubuntu Xenial
containers.

Change-Id: Ieed4a6a3afa6e3ebd9b2f72ba227aac891d65214
Signed-off-by: Pete Birley <pete@port.direct>
2019-06-25 21:14:28 +00:00
Pete Birley
ffb24e337c OpenStack: Check Queens release on Ubuntu Xenial
This PS adds checks for the Queens Release of OpenStack in Ubuntu Xenial
containers.

Change-Id: I0d4d427e43f06fa955dfd275859939d0adca113c
Signed-off-by: Pete Birley <pete@port.direct>
2019-06-25 21:10:26 +00:00
Pete Birley
663328e6e5 Gate: Move jobs with duplicated coverage to periodics
This PS moves the gates we have with duplicated coverage to
periodics, reducing the load we place on -infra with each ps

Change-Id: I03086d05abc8e14dab60494d1126e849f97ee375
Signed-off-by: Pete Birley <pete@port.direct>
2019-06-25 19:40:04 +00:00
Pete Birley
cede6c0d48 Gate: Simpler multiple distro/over-ride support
This PS indroduces a simpler way to incorp over-rides into gate
runs, and also ensures that they are scoped to a single chart, rather
than all of the charts deployed within a gate run.

Depends-On: https://review.opendev.org/666957

Change-Id: I49edf52cc1fc5ec60ee9754c28880c9c0c54492e
Signed-off-by: Pete Birley <pete@port.direct>
2019-06-25 13:16:37 +00:00
Pete Birley
184b3e4326 OpenStack: Check Pike release on Ubuntu Xenial
This PS adds checks for the Pike Release of OpenStack in Ubuntu Xenial
containers.

Change-Id: I402584bbcdd53a4a6bc21f370586b3498142bf81
Signed-off-by: Pete Birley <pete@port.direct>
2019-06-22 11:20:55 +00:00
Steve Wilkerson
0e1c296101 Horizon helm tests
This adds a helm test for Horizon, the helm test
runs a selenium webdriver check to verify the dashboard
is up

Change-Id: I3616c05596b2bd94931c39fb774333bf65453d52
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-06-05 12:41:21 -05:00
Jiří Suchomel
5bb58ebf02 Extend OpenSUSE Leap15 testing
Added more non-voting jobs targeted at OpenSUSE + Rocky.

Depends-On: https://review.opendev.org/#/c/651491/
Depends-On: https://review.opendev.org/#/c/651250/
Depends-On: https://review.opendev.org/#/c/653687/
Depends-On: https://review.opendev.org/#/c/656489/

Change-Id: I679ec679498c896c00b861b945d4f8780f03ec2e
2019-04-30 15:12:18 +02:00
Zuul
5361c3282a Merge "Add OpenSUSE Leap15 testing" 2019-04-26 16:44:16 +00:00
Jean-Philippe Evrard
a828d38316 Add OpenSUSE Leap15 testing
There is currently no testing of the Leap 15 images in OSH.

This addresses it by:
- Using the values_overrides folder according to the multi-os
  spec, creating value override files there for changes that
  needs to happen on Leap 15 images.
- Point to the right images using the previously created folder,
  to allow using those in CI easily.
- Change CI to use previously created overrides.

Depends-On: https://review.openstack.org/#/c/651501
Change-Id: I520d3676195c62b253a19397c86b0d0fbabee710
2019-04-15 11:15:35 +02:00
Steve Wilkerson
0e1dec39d1 Add heat to compute kit and apparmor jobs
This adds the deployment of heat to the compute kit and apparmor
job in order to provide a simple mechanism for booting vms to
validate a functional cloud deployment

Change-Id: I1a0b0fd4fd708a045e82781cfe3990e23d2af581
2019-04-11 20:46:10 +00:00
Steve Wilkerson
0abf9d2b20 Add nonvoting apparmor check job to openstack-helm
This adds a nonvoting apparmor check job to openstack-helm, which
allows for the removal of default apparmor profiles from the nova
chart. This job also includes overrides for using the default
docker apparmor profile for the neutron chart

Change-Id: I8f407f24b7f10c5d7cf10f21f73671f7e6c72767
2019-04-07 06:42:42 +00:00
Pete Birley
91466b6911 Zuul: Add osh nodeset to bandit checks
This ps adds the osh nodeset to bandit checks

Change-Id: I83017db7e47bbfae4e09be08c92a62475750d3d7
Signed-off-by: Pete Birley <pete@port.direct>
2019-03-15 00:29:08 -05:00
Gage Hugo
7406014483 Only run bandit when changing python content
This changes the gate check for bandit to only run when either python
or templated python files are edited. This will cause the check
to only be ran when those specific file types are edited, and to
skip the check when not needed.

Change-Id: I149ef6827ca10e32492f2b05beb8a13af5a03fc5
2019-03-01 19:36:35 +00:00
Steve Wilkerson
61ddb2d87a Add Armada job for testing update of chart passwords
This adds both a periodic and experimental job for deploying Ceph
and the OSH components via Armada. This job will then generate new
passphrases for the OSH components, render an updated manifest for
the OSH components including the new passphrases, then applies the
updated OSH manifest to validate the ability for all deployed
charts to update those passphrases successfully
Change-Id: I42d19bbf8161b60311c4b8101217cdcfbdf6b568
2019-02-06 16:20:16 -06:00
Steve Wilkerson
7c283e7b5d OSH: Add Armada job for testing update of release uuids
This adds both a periodic and experimental job for deploying Ceph
and the OSH components via Armada. This job will then generate a
new release uuid, render an updated manifest for all previously
deployed releases, then apply that manifest to validate the
ability for all deployed charts to update successfully with the
new release uuid annotation

Change-Id: I6f2125f3505904c4714688e7a9900b8d6bea49b4
2019-02-05 08:32:22 -06:00
Zuul
5648754f50 Merge "Update openstack-helm Armada job" 2019-02-04 20:17:16 +00:00
Steve Wilkerson
dc1ef4c138 Update openstack-helm Armada job
This updates the openstack-helm Armada job to instead deploy
only Ceph, the OpenStack service charts, and their dependencies.
This is dependent on the addition of the Armada job for Ceph and
the LMA components to openstack-helm-infra. This also updates the
jobs definition to use the osh-gate-runner playbook instead, as
well as sets the job both to a periodic and experimental job

Depends-On: https://review.openstack.org/#/c/634676/
Depends-On: https://review.openstack.org/#/c/633067/
Change-Id: I7e191a153f123e04e123acc33fb691d8117062a9
2019-02-04 16:46:22 +00:00
Gage Hugo
482dbcac29 Scan templated python files with bandit
This change adds a zuul check job to export any templated python
contained in the helm charts and scan it with bandit for any
potential security flaws.

This also adds two nosec comments on the instances of subprocess
used as they currently do not appear to be malicious, as well
as changing the endpoint_update python code to prevent sql
injection, which satisfies bandit code B608.

Change-Id: I2212d26514c3510353d16a4592893dd2e85cb369
2019-01-22 20:53:52 +00:00
Steve Wilkerson
f546dad007 Reinstate openstack-helm checks
A change was merged that had commented out the check jobs. This
simply uncomments them so checks run against changes to
openstack-helm

The change can be found here: https://review.openstack.org/#/c/591808/48

Change-Id: Ia100f1248ebe783d154420c543a9b19fb1ba4ccc
2019-01-02 11:27:58 -06:00