charm cleanup

Following changes are done as part of this patch:

rename sunbeam import library to ops_sunbeam
update documentation
add fetch-libs in tox.ini
add ingress-public interface

charms/cinder-k8s/CONTRIBUTING.md

@@ -1,4 +1,4 @@
-# charm-cinder-operator
+# cinder-k8s
 
 ## Developing
 
@@ -10,21 +10,26 @@ Create and activate a virtualenv with the development requirements:
 
 ## Code overview
 
-TEMPLATE-TODO: 
+Get familiarise with [Charmed Operator Framework](https://juju.is/docs/sdk)
-One of the most important things a consumer of your charm (or library)
+and [Sunbeam documentation](sunbeam-docs).
-needs to know is what set of functionality it provides. Which categories
+
-does it fit into? Which events do you listen to? Which libraries do you
+cinder-k8s charm uses the ops_sunbeam library and extends
-consume? Which ones do you export and how are they used?
+OSBaseOperatorAPICharm from the library.
+
+cinder-k8s charm consumes shared-db relation to connect to database,
+identity-service to register the service endpoints to keystone
+and ingress-internal/ingress-public relation to get exposed over
+internal and public networks.
 
 ## Intended use case
 
-TEMPLATE-TODO:
+cinder-k8s charm deploys and configures OpenStack Block storage service
-Why were these decisions made? What's the scope of your charm?
+on a kubernetes based environment. It should be possible to use
+local storage or ceph based storage as backend.
 
 ## Roadmap
 
-If this Charm doesn't fulfill all of the initial functionality you were
+TODO
-hoping for or planning on, please add a Roadmap or TODO here
 
 ## Testing
 
@@ -32,3 +37,20 @@ The Python operator framework includes a very nice harness for testing
 operator behaviour without full deployment. Just `run_tests`:
 
     ./run_tests
+
+## Deployment
+
+This project uses tox for building and managing. To build the charm
+run:
+
+    tox -e build
+
+To deploy the local test instance:
+
+    tox -e build
+    juju add-model cinder
+    juju deploy ./cinder-k8s_ubuntu-20.04-amd64.charm --resource cinder-api-image=kolla/ubuntu-binary-cinder-api:xena --resource cinder-scheduler-image=kolla/ubuntu-binary-cinder-scheduler:xena
+
+<!-- LINKS -->
+
+[sunbeam-docs]: https://github.com/openstack-charmers/advanced-sunbeam-openstack/blob/main/README.rst

charms/cinder-k8s/README.md

@@ -1,24 +1,69 @@
-# charm-cinder-operator
+# cinder-k8s
 
 ## Description
 
-TODO: Describe your charm in a few paragraphs of Markdown
+The cinder-k8s is an operator to manage the cinder service on a
+kubernetes based environment.
 
 ## Usage
 
-TODO: Provide high-level usage, such as required config or relations
+### Deployment
 
+cinder-k8s is deployed using below command:
+
+    juju deploy cinder-k8s cinder --trust
+
+Now connect the cinder application to an existing database,
+amqp and keystone identity.
+
+    juju relate mysql:database cinder:shared-db
+    juju relate rabbitmq:amqp cinder:amqp
+    juju relate keystone:identity-service cinder:identity-service
+
+### Configuration
+
+This section covers common and/or important configuration options. See file
+`config.yaml` for the full list of options, along with their descriptions and
+default values. See the [Juju documentation][juju-docs-config-apps] for details
+on configuring applications.
+
+### Actions
+
+This section covers Juju [actions][juju-docs-actions] supported by the charm.
+Actions allow specific operations to be performed on a per-unit basis. To
+display action descriptions run `juju actions cinder`. If the charm is not
+deployed then see file `actions.yaml`.
 
 ## Relations
 
-TODO: Provide any relations which are provided or required by your charm
+cinder-k8s requires the following relations:
+
+`shared-db`: To connect to the database
+`amqp`: To connect to rabbitmq
+`identity-service`: To register endpoints in keystone
+`ingress-internal`: To expose service on underlying internal network
+`ingress-public`: To expose service on public network
+`storage-backend`: To connect to backend which exposes storage
 
 ## OCI Images
 
-TODO: Include a link to the default image your charm uses
+The charm by default uses follwoing images:
+`docker.io/kolla/ubuntu-binary-cinder-api:xena`
+`docker.io/kolla/ubuntu-binary-cinder-scheduler:xena`
 
 ## Contributing
 
 Please see the [Juju SDK docs](https://juju.is/docs/sdk) for guidelines
 on enhancements to this charm following best practice guidelines, and
-`CONTRIBUTING.md` for developer guidance.
+[CONTRIBUTING.md](contributors-guide) for developer guidance.
+
+## Bugs
+
+Please report bugs on [Launchpad][lp-bugs-charm-cinder-k8s].
+
+<!-- LINKS -->
+
+[contributors-guide]: https://github.com/openstack-charmers/charm-cinder-operator/blob/main/CONTRIBUTING.md
+[juju-docs-actions]: https://jaas.ai/docs/actions
+[juju-docs-config-apps]: https://juju.is/docs/configuring-applications
+[lp-bugs-charm-cinder-k8s]: https://bugs.launchpad.net/charm-cinder-k8s/+filebug

charms/cinder-k8s/fetch-libs.sh

@@ -0,0 +1,9 @@
+#!/bin/bash
+
+echo "INFO: Fetching libs from charmhub."
+charmcraft fetch-lib charms.nginx_ingress_integrator.v0.ingress
+charmcraft fetch-lib charms.sunbeam_mysql_k8s.v0.mysql
+charmcraft fetch-lib charms.sunbeam_keystone_operator.v0.identity_service
+charmcraft fetch-lib charms.sunbeam_rabbitmq_operator.v0.amqp
+charmcraft fetch-lib charms.observability_libs.v0.kubernetes_service_patch
+charmcraft fetch-lib charms.traefik_k8s.v0.ingress

charms/cinder-k8s/lib/charms/sunbeam_keystone_operator/v0/identity_service.py

@@ -26,7 +26,7 @@ Two events are also available to respond to:
 A basic example showing the usage of this relation follows:
 
 ```
-from charms.sunbeam_sunbeam_identity_service_operator.v0.identity_service import IdentityServiceRequires
+from charms.sunbeam_keystone_operator.v0.identity_service import IdentityServiceRequires
 
 class IdentityServiceClientCharm(CharmBase):
     def __init__(self, *args):
@@ -309,6 +309,20 @@ class IdentityServiceRequires(Object):
         """Return the service_user_id."""
         return self.get_remote_app_data('service-user-id')
 
+    @property
+    def internal_auth_url(self) -> str:
+        """Return the internal_auth_url."""
+        return self.get_remote_app_data('internal-auth-url')
+
+    @property
+    def admin_auth_url(self) -> str:
+        """Return the admin_auth_url."""
+        return self.get_remote_app_data('admin-auth-url')
+
+    @property
+    def public_auth_url(self) -> str:
+        """Return the public_auth_url."""
+        return self.get_remote_app_data('public-auth-url')
 
     def register_services(self, service_endpoints: dict,
                           region: str) -> None:
@@ -439,7 +453,10 @@ class IdentityServiceProvides(Object):
                                          service_domain: str,
                                          service_password: str,
                                          service_project: str,
-                                         service_user: str):
+                                         service_user: str,
+                                         internal_auth_url: str,
+                                         admin_auth_url: str,
+                                         public_auth_url: str):
         logging.debug("Setting identity_service connection information.")
         for relation in self.framework.model.relations[relation_name]:
             if relation.id == relation_id:
@@ -468,3 +485,6 @@ class IdentityServiceProvides(Object):
         app_data["service-user-name"] = service_user.name
         app_data["service-user-id"] = service_user.id
         app_data["service-password"] = service_password
+        app_data["internal-auth-url"] = internal_auth_url
+        app_data["admin-auth-url"] = admin_auth_url
+        app_data["public-auth-url"] = public_auth_url

charms/cinder-k8s/metadata.yaml

@@ -1,6 +1,6 @@
 # Copyright 2021 Canonical Ltd
 # See LICENSE file for licensing details.
-name: sunbeam-cinder-operator
+name: cinder-k8s
 summary: OpenStack volume service
 maintainer: Openstack Charmers <openstack-charmers@lists.ubuntu.com>
 description: |
@@ -35,9 +35,13 @@ requires:
   shared-db:
     interface: mysql_datastore
     limit: 1
-  ingress:
+  ingress-internal:
     interface: ingress
     limit: 1
+  ingress-public:
+    interface: ingress
+    optional: true
+    limit: 1
   identity-service:
     interface: keystone
     limit: 1

charms/cinder-k8s/requirements.txt

@@ -3,10 +3,9 @@ jinja2
 # Get resources from github until cacerts issue is charmbuild image is fixed.
 # git+https://opendev.org/openstack/charm-ops-openstack#egg=ops_openstack
 # git+https://opendev.org/openstack/charm-ops-interface-tls-certificates#egg=interface_tls_certificates
-git+https://github.com/openstack/charm-ops-openstack#egg=ops_openstack
 git+https://github.com/openstack/charm-ops-interface-tls-certificates#egg=interface_tls_certificates
 
-git+https://github.com/openstack-charmers/advanced-sunbeam-openstack#egg=advanced_sunbeam_openstack
+git+https://github.com/openstack-charmers/advanced-sunbeam-openstack#egg=ops_sunbeam
 lightkube
 lightkube-models
 cryptography < 3.4

charms/cinder-k8s/run_tests

@@ -1,5 +1,5 @@
 #!/bin/sh -e
-# Copyright 2021 James Page
+# Copyright 2021 Canonical Ltd.
 # See LICENSE file for licensing details.
 
 if [ -z "$VIRTUAL_ENV" -a -d venv/ ]; then

charms/cinder-k8s/src/charm.py

@@ -12,10 +12,10 @@ import ops.pebble
 from ops.framework import StoredState
 from ops.main import main
 
-import advanced_sunbeam_openstack.charm as sunbeam_charm
+import ops_sunbeam.charm as sunbeam_charm
-import advanced_sunbeam_openstack.core as sunbeam_core
+import ops_sunbeam.core as sunbeam_core
-import advanced_sunbeam_openstack.container_handlers as sunbeam_chandlers
+import ops_sunbeam.container_handlers as sunbeam_chandlers
-import advanced_sunbeam_openstack.relation_handlers as sunbeam_rhandlers
+import ops_sunbeam.relation_handlers as sunbeam_rhandlers
 
 import charms.sunbeam_cinder_operator.v0.storage_backend as sunbeam_storage_backend  # noqa
 
@@ -182,6 +182,21 @@ class CinderOperatorCharm(sunbeam_charm.OSBaseOperatorAPICharm):
             },
         ]
 
+    @property
+    def container_configs(self) -> List[sunbeam_core.ContainerConfigFile]:
+        """Container configuration files for the service."""
+        _cconfigs = super().container_configs
+        _cconfigs.extend(
+            [
+                sunbeam_core.ContainerConfigFile(
+                    '/etc/cinder/api-paste.ini',
+                    self.service_user,
+                    self.service_group,
+                )
+            ]
+        )
+        return _cconfigs
+
     def get_pebble_handlers(self):
         pebble_handlers = [
             CinderWSGIPebbleHandler(

charms/cinder-k8s/src/templates/api-paste.ini.j2

@@ -0,0 +1,61 @@
+[composite:osapi_volume]
+use = call:cinder.api:root_app_factory
+/: apiversions
+/healthcheck: healthcheck
+/v3: openstack_volume_api_v3
+{% if ingress_internal.ingress_path -%}
+{{ ingress_internal.ingress_path }}: apiversions
+{{ ingress_internal.ingress_path }}/v3: openstack_volume_api_v3
+{% endif -%}
+
+[composite:openstack_volume_api_v3]
+use = call:cinder.api.middleware.auth:pipeline_factory
+noauth = cors http_proxy_to_wsgi request_id faultwrap sizelimit osprofiler noauth apiv3
+keystone = cors http_proxy_to_wsgi request_id faultwrap sizelimit osprofiler authtoken keystonecontext apiv3
+keystone_nolimit = cors http_proxy_to_wsgi request_id faultwrap sizelimit osprofiler authtoken keystonecontext apiv3
+
+[filter:request_id]
+paste.filter_factory = oslo_middleware.request_id:RequestId.factory
+
+[filter:http_proxy_to_wsgi]
+paste.filter_factory = oslo_middleware.http_proxy_to_wsgi:HTTPProxyToWSGI.factory
+
+[filter:cors]
+paste.filter_factory = oslo_middleware.cors:filter_factory
+oslo_config_project = cinder
+
+[filter:faultwrap]
+paste.filter_factory = cinder.api.middleware.fault:FaultWrapper.factory
+
+[filter:osprofiler]
+paste.filter_factory = osprofiler.web:WsgiMiddleware.factory
+
+[filter:noauth]
+paste.filter_factory = cinder.api.middleware.auth:NoAuthMiddleware.factory
+
+[filter:sizelimit]
+paste.filter_factory = oslo_middleware.sizelimit:RequestBodySizeLimiter.factory
+
+[app:apiv3]
+paste.app_factory = cinder.api.v3.router:APIRouter.factory
+
+[pipeline:apiversions]
+pipeline = cors http_proxy_to_wsgi faultwrap osvolumeversionapp
+
+[app:osvolumeversionapp]
+paste.app_factory = cinder.api.versions:Versions.factory
+
+##########
+# Shared #
+##########
+
+[filter:keystonecontext]
+paste.filter_factory = cinder.api.middleware.auth:CinderKeystoneContext.factory
+
+[filter:authtoken]
+paste.filter_factory = keystonemiddleware.auth_token:filter_factory
+
+[app:healthcheck]
+paste.app_factory = oslo_middleware:Healthcheck.app_factory
+backends = disable_by_file
+disable_by_file_path = /etc/cinder/healthcheck_disable

charms/cinder-k8s/src/templates/cinder.conf.j2

@@ -12,15 +12,7 @@ auth_strategy = keystone
 
 {% include "parts/section-database" %}
 
-[keystone_authtoken]
+{% include "parts/section-identity" %}
-www_authenticate_uri = {{ identity_service.internal_protocol }}://{{ identity_service.internal_host }}:{{ identity_service.internal_port }}
-auth_url = {{ identity_service.internal_protocol }}://{{ identity_service.internal_host }}:{{ identity_service.internal_port }}
-auth_type = password
-project_domain_name = {{ identity_service.service_domain_name }}
-user_domain_name = {{ identity_service.service_domain_name }}
-project_name = {{ identity_service.service_project_name }}
-username = {{ identity_service.service_user_name }}
-password = {{ identity_service.service_password }}
 
 [oslo_concurrency]
 lock_path = /var/lib/cinder/tmp

charms/cinder-k8s/src/templates/parts/section-identity

@@ -0,0 +1,14 @@
+[keystone_authtoken]
+{% if identity_service.internal_auth_url -%}
+www_authenticate_uri = {{ identity_service.internal_auth_url }}
+auth_url = {{ identity_service.internal_auth_url }}
+{% elif identity_service.internal_host -%}
+www_authenticate_uri = {{ identity_service.internal_protocol }}://{{ identity_service.internal_host }}:{{ identity_service.internal_port }}
+auth_url = {{ identity_service.internal_protocol }}://{{ identity_service.internal_host }}:{{ identity_service.internal_port }}
+{% endif -%}
+auth_type = password
+project_domain_name = {{ identity_service.service_domain_name }}
+user_domain_name = {{ identity_service.service_domain_name }}
+project_name = {{ identity_service.service_project_name }}
+username = {{ identity_service.service_user_name }}
+password = {{ identity_service.service_password }}

charms/cinder-k8s/tests/test_charm.py

@@ -1,4 +1,4 @@
-# Copyright 2021 James Page
+# Copyright 2021 Canonical Ltd.
 # See LICENSE file for licensing details.
 #
 # Learn more about testing at: https://juju.is/docs/sdk/testing

charms/cinder-k8s/tox.ini

@@ -18,7 +18,7 @@ skip_missing_interpreters = False
 requires = pip < 20.3
            virtualenv < 20.0
 # NOTE: https://wiki.canonical.com/engineering/OpenStack/InstallLatestToxOnOsci
-minversion = 3.2.0
+minversion = 3.18.0
 
 [testenv]
 setenv = VIRTUAL_ENV={envdir}
@@ -35,6 +35,12 @@ whitelist_externals =
 passenv = HOME TERM CS_* OS_* TEST_*
 deps = -r{toxinidir}/test-requirements.txt
 
+[testenv:fetch]
+basepython = python3
+deps =
+commands =
+  ./fetch-libs.sh
+
 [testenv:py3.8]
 basepython = python3.8
 deps = -r{toxinidir}/requirements.txt

charms/cinder-k8s/unit_tests/test_cinder_charm.py

@@ -21,7 +21,7 @@ sys.path.append('lib')  # noqa
 sys.path.append('src')  # noqa
 
 import charm
-import advanced_sunbeam_openstack.test_utils as test_utils
+import ops_sunbeam.test_utils as test_utils
 
 
 class _CinderXenaOperatorCharm(charm.CinderXenaOperatorCharm):