Add wsgi module to Octavia and remove wsgi script

Changes in python packaging tooling mean that the wsgi_scripts
functionality via PBR may not longer function.

This patch switches Octavia from using the PBR wsgi_scripts method to
using a new wsgi module that provides the same behavior as the generated
wsgi scripts provided.

A related devstack patch enables devstack to setup uWSGI to use the new
module path instead of the generated wsgi scripts.

This also aligns Octavia to a new proposed OpenStack goal[1].

[1] https://review.opendev.org/c/openstack/governance/+/902807

Depends-On: https://review.opendev.org/c/openstack/devstack/+/902758

Closes-Bug: #2109665

In order to make the CI work, this patch was merged with:

zuul: Drop centos9/py39 and make grenade and barbican non-voting

The requirements repo has dropped upper-constraints for python3.9, so
the centos9 and py39 based jobs are no longer working on master, let's
drop them.

The grenade jobs are broken due to the #2109665, make them
temporarily non-voting, so that we can merge the fix[0] on master first,
backport it and then re-enable them.

Finally, octavia-v2-dsvm-tls-barbican fails due to #2109584, so a fix
similar to [0] will be needed for barbican.

[0] https://review.opendev.org/c/openstack/octavia/+/902812

Related-Bug: #2109665
Related-Bug: #2109584

Change-Id: I54f8144a3347c3c5bf2e2e99e9d500a0c6fb89eb

devstack/plugin.sh

@@ -185,7 +185,7 @@ function _configure_octavia_apache_wsgi {
 }
 
 function _configure_octavia_apache_uwsgi {
-    write_uwsgi_config "$OCTAVIA_UWSGI_CONF" "$OCTAVIA_UWSGI_APP" "/$OCTAVIA_SERVICE_TYPE"
+    write_uwsgi_config "$OCTAVIA_UWSGI_CONF" "$OCTAVIA_UWSGI_APP" "/$OCTAVIA_SERVICE_TYPE" "" "octavia-wsgi"
 }
 
 

devstack/settings

@@ -111,5 +111,5 @@ OCTAVIA_MGMT_PORT_IP=${OCTAVIA_MGMT_PORT_IP:-"auto"}
 OCTAVIA_DIB_TRACING=${OCTAVIA_DIB_TRACING:-"1"}
 
 OCTAVIA_SERVICE_TYPE="load-balancer"
-OCTAVIA_UWSGI_APP=${OCTAVIA_UWSGI_APP:-${OCTAVIA_BIN_DIR}/octavia-wsgi}
+OCTAVIA_UWSGI_APP=${OCTAVIA_UWSGI_APP:-octavia.wsgi.api:application}
 OCTAVIA_UWSGI_CONF=${OCTAVIA_UWSGI_CONF:-${OCTAVIA_CONF_DIR}/octavia-uwsgi.ini}

octavia/wsgi/api.py

@@ -0,0 +1,34 @@
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+"""WSGI application entry-point for the Octavia API."""
+
+from pathlib import Path
+from sys import argv
+import threading
+
+from octavia.api import app
+
+application = None
+args = None
+
+# Our wsgi app will pull in the sys.argv if we don't pass an argv parameter
+# which means octavia will try to use the sphinx-build parameters for oslo
+# config. Work around this while maintaining compatiblity by passing the 'h'
+# help parameter if we are "running" under sphinx-build.
+if Path(argv[0]).name == "sphinx-build":
+    args = ['h']
+
+lock = threading.Lock()
+with lock:
+    if application is None:
+        application = app.setup_app(argv=args)

releasenotes/notes/remove-wsgi-scripts-a66048263bd550c6.yaml

@@ -0,0 +1,29 @@
+---
+features:
+  - |
+    A new module, ``octavia.wsgi``, has been added as a place to gather WSGI
+    ``application`` objects. This is intended to ease deployment by providing
+    a consistent location for these objects. For example, if using uWSGI then
+    instead of:
+
+    .. code-block:: ini
+
+        [uwsgi]
+        wsgi-file = /bin/octavia-wsgi
+
+    You can now use:
+
+    .. code-block:: ini
+
+        [uwsgi]
+        module = octavia.wsgi.api:application
+
+    This also simplifies deployment with other WSGI servers that expect module
+    paths such as gunicorn.
+upgrade:
+  - |
+    The WSGI script ``octavia-wsgi`` has been removed. Deployment tooling
+    should instead reference the Python module path for the wsgi module in
+    Octavia, ``octavia.wsgi.api:application`` if their chosen WSGI server
+    supports this (gunicorn, uWSGI, etc.) or implement a .wsgi script
+    themselves if not (mod_wsgi).

setup.cfg

@@ -38,8 +38,6 @@ data_files =
         diskimage-create/version.txt
 
 [entry_points]
-wsgi_scripts =
-    octavia-wsgi = octavia.api.app:setup_app
 console_scripts =
     octavia-api = octavia.cmd.api:main
     octavia-worker = octavia.cmd.octavia_worker:main

zuul.d/jobs.yaml

@@ -244,21 +244,6 @@
         OCTAVIA_AMP_IMAGE_SIZE: 3
         OCTAVIA_AMP_ENABLE_FIPS: True
 
-- job:
-    name: octavia-v2-dsvm-tls-barbican-fips
-    parent: octavia-v2-dsvm-tls-barbican
-    nodeset: octavia-single-node-centos-9-stream
-    description: |
-      Functional testing for a FIPS enabled Centos 9 system
-    pre-run: playbooks/enable-fips.yaml
-    vars:
-      nslookup_target: 'opendev.org'
-      devstack_localrc:
-        OCTAVIA_AMP_BASE_OS: centos
-        OCTAVIA_AMP_DISTRIBUTION_RELEASE_ID: 9-stream
-        OCTAVIA_AMP_IMAGE_SIZE: 3
-        OCTAVIA_AMP_ENABLE_FIPS: True
-
 - job:
     name: octavia-v2-dsvm-scenario-traffic-ops-ubuntu-jammy
     parent: octavia-v2-dsvm-scenario-ubuntu-jammy

zuul.d/projects.yaml

@@ -19,7 +19,7 @@
               - ^etc/.*$
               - ^octavia/tests/.*$
               - ^releasenotes/.*$
-        - openstack-tox-functional-py39:
+        - openstack-tox-functional-py310:
             irrelevant-files: &tox-func-irrelevant-files
               - ^.*\.rst$
               - ^api-ref/.*$
@@ -27,8 +27,6 @@
               - ^etc/.*$
               - ^octavia/tests/unit/.*$
               - ^releasenotes/.*$
-        - openstack-tox-functional-py310:
-            irrelevant-files: *tox-func-irrelevant-files
         - openstack-tox-functional-py311:
             irrelevant-files: *tox-func-irrelevant-files
         # Undefined yet:
@@ -56,6 +54,7 @@
             irrelevant-files: *irrelevant-files
         - octavia-v2-dsvm-tls-barbican:
             irrelevant-files: *irrelevant-files
+            voting: false  # due to LP#948391
         - octavia-grenade:
             irrelevant-files: &grenade-irrelevant-files
               - ^.*\.rst$
@@ -67,12 +66,10 @@
               - ^tools/.*$
               - ^(test-|)requirements.txt$
               - ^tox.ini$
+            voting: false  # due to LP#2109665
         - octavia-grenade-skip-level:
             irrelevant-files: *grenade-irrelevant-files
             voting: false
-        - octavia-v2-dsvm-tls-barbican-fips:
-            irrelevant-files: *irrelevant-files
-            voting: false
         - octavia-v2-act-stdby-dsvm-scenario:
             irrelevant-files: *irrelevant-files
             voting: false
@@ -85,9 +82,6 @@
         - octavia-v2-dsvm-scenario-ipv6-only:
             irrelevant-files: *irrelevant-files
             voting: false
-        - octavia-v2-dsvm-scenario-centos-9-stream:
-            irrelevant-files: *irrelevant-files
-            voting: false
     queue: octavia
     gate:
       fail-fast: true
@@ -113,8 +107,10 @@
         - octavia-v2-dsvm-scenario-non-traffic-ops
         - octavia-v2-dsvm-scenario-traffic-ops-ubuntu-jammy
         - octavia-v2-dsvm-scenario-non-traffic-ops-ubuntu-jammy
-        - octavia-v2-dsvm-tls-barbican
-        - octavia-grenade
+        - octavia-v2-dsvm-tls-barbican:
+            voting: false
+        - octavia-grenade:
+            voting: false
         #- octavia-grenade-skip-level
     periodic:
       jobs: