From 589803c186697d1a2212422c48d5acf8e767dc7f Mon Sep 17 00:00:00 2001 From: James Kirsch Date: Thu, 2 Jul 2020 13:40:37 -0700 Subject: [PATCH] Update TLS documentation Updated TLS documentation to reflect new features and configuration options added in Ussuri. Change-Id: I74550eaf394287b14fc521293cc4b5ea8074192c Partially-Implements: blueprint add-ssl-internal-network --- doc/source/admin/tls.rst | 371 +++++++++++++++++++++++++++++---------- 1 file changed, 282 insertions(+), 89 deletions(-) diff --git a/doc/source/admin/tls.rst b/doc/source/admin/tls.rst index eae7241fe8..eefca82477 100644 --- a/doc/source/admin/tls.rst +++ b/doc/source/admin/tls.rst @@ -2,127 +2,320 @@ TLS === -An additional endpoint configuration option is to enable or disable -TLS protection for the internal and/or external VIP. TLS allows a client to -authenticate the OpenStack service endpoint and allows for encryption of the -requests and responses. +This guide describes how to configure Kolla Ansible to deploy OpenStack with +TLS enabled. Enabling TLS on the provided internal and/or external VIP address +allows OpenStack clients to authenticate and encrypt network communication +with OpenStack services. -The configuration variables that control TLS networking are: +When an OpenStack service exposes an API endpoint, Kolla Ansible will +configure HAProxy for that service to listen on the internal and/or +external VIP address. The HAProxy container load-balances requests on +the VIPs to the nodes running the service container. -- kolla_enable_tls_external -- kolla_external_fqdn_cert -- kolla_enable_tls_internal -- kolla_internal_fqdn_cert +There are two different layers of TLS configuration for OpenStack APIs: + +1. Enabling TLS on the internal and/or external VIP, so communication between + an OpenStack client and the HAProxy listening on the VIP is secure. +2. Enabling TLS on the backend network, so communication between HAProxy and + the backend API services is secure. .. note:: - If TLS is enabled only on the internal or the external network - the kolla_internal_vip_address and kolla_external_vip_address must - be different. + TLS authentication is based on certificates that have been signed by + trusted Certificate Authorities. Examples of commercial CAs are Comodo, + Symantec, GoDaddy, and GlobalSign. Letsencrypt.org is a CA that will + provide trusted certificates at no charge. If using a trusted CA is not + possible for your project, you can use a private CA, e.g. Hashicorp Vault, + to create a certificate for your domain, or see + :ref:`admin-tls-generating-a-private-ca` to use a Kolla Ansible generated + private CA. - If there is only a single network configured in your network topology - (opposed to configuring seperate internal and external networks), TLS - can be enabled using only the internal network configuration variables. +Quick Start +~~~~~~~~~~~ -The default for TLS is disabled, to enable TLS networking: +.. note:: + + The certificates generated by Kolla Ansible use a simple Certificate + Authority setup and are not suitable for a production deployment. Only + certificates signed by a trusted Certificate Authority should be used in a + production deployment. + +To deploy OpenStack with TLS enabled for the external, internal and backend +APIs, configure the following in ``globals.yml``: .. code-block:: yaml - kolla_enable_tls_external: "yes" - kolla_external_fqdn_cert: "{{ kolla_certificates_dir }}/mycert.pem" + kolla_enable_tls_internal: "yes" + kolla_enable_tls_external: "yes" + kolla_enable_tls_backend: "yes" + kolla_copy_ca_into_containers: "yes" - and/or +If deploying on Debian or Ubuntu: - kolla_enable_tls_internal: "yes" - kolla_internal_fqdn_cert: "{{ kolla_certificates_dir }}/mycert-internal.pem" +.. code-block:: yaml + openstack_cacert: "/etc/ssl/certs/ca-certificates.crt" -.. note:: +If on CentOS or RHEL: - TLS authentication is based on certificates that have been - signed by trusted Certificate Authorities. Examples of commercial - CAs are Comodo, Symantec, GoDaddy, and GlobalSign. Letsencrypt.org - is a CA that will provide trusted certificates at no charge. Many - company's IT departments will provide certificates within that - company's domain. If using a trusted CA is not possible for your - situation, you can use `OpenSSL `__ - to create your own company's domain or see the section below about - kolla generated self-signed certificates. +.. code-block:: yaml -Two certificate files are required to use TLS securely with authentication. -These two files will be provided by your Certificate Authority. These -two files are the server certificate with private key and the CA certificate -with any intermediate certificates. The server certificate needs to be -installed with the kolla deployment and is configured with the -``kolla_external_fqdn_cert`` or ``kolla_internal_fqdn_cert`` parameter. -If the server certificate provided is not already trusted by the client, -then the CA certificate file will need to be distributed to the client. + openstack_cacert: "/etc/pki/tls/certs/ca-bundle.crt" -When using TLS to connect to a public endpoint, an OpenStack client will -have settings similar to this: - -.. code-block:: shell - - export OS_PROJECT_DOMAIN_ID=default - export OS_USER_DOMAIN_ID=default - export OS_PROJECT_NAME=demo - export OS_USERNAME=demo - export OS_PASSWORD=demo-password - export OS_AUTH_URL=https://mykolla.example.net:5000 - # os_cacert is optional for trusted certificates - export OS_CACERT=/etc/pki/ca/mykolla-cacert.crt - export OS_IDENTITY_API_VERSION=3 - -Self-Signed Certificates -~~~~~~~~~~~~~~~~~~~~~~~~ - -.. note:: - - Self-signed certificates should never be used in production. - -It is not always practical to get a certificate signed by a well-known -trust CA, for example a development or internal test kolla deployment. In -these cases it can be useful to have a self-signed certificate to use. - -For convenience, the ``kolla-ansible`` command will generate the necessary -certificate files based on the information in the ``globals.yml`` -configuration file: +The Kolla Ansible ``certificates`` command generates a private test Certificate +Authority, and then uses the CA to sign the generated certificates for the +enabled VIP(s) to test TLS in your OpenStack deployment. Assuming you are +using the ``multinode`` inventory: .. code-block:: console - kolla-ansible certificates + kolla-ansible -i ~/multinode certificates -The certificate file haproxy.pem will be generated and stored in the -``/etc/kolla/certificates/`` directory, and the CA cert will be in the -``/etc/kolla/certificates/ca/`` directory. +TLS Configuration for internal/external VIP +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The configuration variables that control TLS for the internal and/or external +VIP are: + +- ``kolla_enable_tls_external`` +- ``kolla_enable_tls_internal`` +- ``kolla_internal_fqdn_cert`` +- ``kolla_external_fqdn_cert`` + +.. note:: + + If TLS is enabled only on the internal or external network, then + ``kolla_internal_vip_address`` and ``kolla_external_vip_address`` must + be different. + + If there is only a single network configured in your topology + (as opposed to separate internal and external networks), TLS + can only be enabled using the internal network configuration variables. + +The default state for TLS networking is disabled. To enable external TLS +encryption: + +.. code-block:: yaml + + kolla_enable_tls_external: "yes" + +To enable internal TLS encryption: + +.. code-block:: yaml + + kolla_enable_tls_internal: "yes" + +Two certificate files are required to use TLS securely with authentication, +which will be provided by your Certificate Authority: + +* server certificate with private key +* CA certificate with any intermediate certificates + +The combined server certificate and private key needs to be provided to Kolla +Ansible, with the path configured via ``kolla_external_fqdn_cert`` or +``kolla_internal_fqdn_cert``. These paths default to ``{{ +kolla_certificates_dir }}/haproxy.pem`` and ``{{ kolla_certificates_dir +}}/haproxy-internal.pem`` respectively, where ``kolla_certificates_dir`` is +``/etc/kolla/certificates`` by default. + +If the server certificate provided is not already trusted by clients, then +the CA certificate file will need to be distributed to the clients. This is +discussed in more detail in :ref:`admin-tls-openrc` and +:ref:`admin-tls-ca-in-containers`. + +.. _admin-tls-openrc: + +Configuring the OpenStack Client for TLS +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The location for the CA certificate for the ``admin-openrc.sh`` file is +configured with the ``kolla_admin_openrc_cacert`` variable, which is not set by +default. This must be a valid path on all hosts where ``admin-openrc.sh`` is +used. + +When TLS is enabled on a VIP, and ``kolla_admin_openrc_cacert`` is set to +``/etc/pki/tls/certs/ca-bundle.crt``, an OpenStack client will have settings +similar to this configured by ``admin-openrc.sh``: + +.. code-block:: shell + + export OS_PROJECT_DOMAIN_NAME=Default + export OS_USER_DOMAIN_NAME=Default + export OS_PROJECT_NAME=admin + export OS_TENANT_NAME=admin + export OS_USERNAME=admin + export OS_PASSWORD=demoPassword + export OS_AUTH_URL=https://mykolla.example.net:5000 + export OS_INTERFACE=internal + export OS_ENDPOINT_TYPE=internalURL + export OS_MISTRAL_ENDPOINT_TYPE=internalURL + export OS_IDENTITY_API_VERSION=3 + export OS_REGION_NAME=RegionOne + export OS_AUTH_PLUGIN=password + # os_cacert is optional for trusted certificates + export OS_CACERT=/etc/pki/tls/certs/ca-bundle.crt + +.. _admin-tls-ca-in-containers: Adding CA Certificates to the Service Containers ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -To copy CA certificate files to the service containers +To copy CA certificate files to the service containers: .. code-block:: yaml - kolla_copy_ca_into_containers: "yes" + kolla_copy_ca_into_containers: "yes" -When ``kolla_copy_ca_into_containers`` is configured to "yes", the -CA certificate files in /etc/kolla/certificates/ca will be copied into +When ``kolla_copy_ca_into_containers`` is configured to "yes", the CA +certificate files in ``/etc/kolla/certificates/ca`` will be copied into service containers to enable trust for those CA certificates. This is required for any certificates that are either self-signed or signed by a private CA, -and are not already present in the service image trust store. +and are not already present in the service image trust store. Kolla will +install these certificates in the container system wide trust store when the +container starts. -All certificate file names will have the "kolla-customca-" prefix prepended to -it when it is copied into the containers. For example, if a certificate file is -named "internal.crt", it will be named "kolla-customca-internal.crt" in the -containers. +All certificate file names will have the ``kolla-customca-`` prefix prepended +to them when they are copied into the containers. For example, if a certificate +file is named ``internal.crt``, it will be named +``kolla-customca-internal.crt`` in the containers. -For Debian and Ubuntu containers, the certificate files will be copied to -the ``/usr/local/share/ca-certificates/`` directory. +For Debian and Ubuntu containers, the certificate files will be copied to the +``/usr/local/share/ca-certificates/`` directory. -For Centos and Red Hat Linux containers, the certificate files will be copied -to the ``/etc/pki/ca-trust/source/anchors/`` directory. +For CentOS and RHEL containers, the certificate files will be copied to the +``/etc/pki/ca-trust/source/anchors/`` directory. -In addition, the ``openstack_cacert`` should be configured with the path to -the cacert in the container. For example, if the self-signed certificate task -was used and the deployment is on ubuntu, the path would be: -"/etc/pki/ca-trust/source/anchors/kolla-customca-haproxy-internal.crt" +In both cases, valid certificates will be added to the system trust store - +``/etc/ssl/certs/ca-certificates.crt`` on Debian and Ubuntu, and +``/etc/pki/tls/certs/ca-bundle.crt`` on CentOS and RHEL. + +Configuring a CA bundle +~~~~~~~~~~~~~~~~~~~~~~~ + +OpenStack services do not always trust CA certificates from the system trust +store by default. To resolve this, the ``openstack_cacert`` variable should be +configured with the path to the CA Certificate in the container. + +To use the system trust store on Debian or Ubuntu: + +.. code-block:: yaml + + openstack_cacert: /etc/ssl/certs/ca-certificates.crt + +For CentOS or RHEL: + +.. code-block:: yaml + + openstack_cacert: /etc/pki/tls/certs/ca-bundle.crt + +Back-end TLS Configuration +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Enabling TLS on the backend services secures communication between the +HAProxy listing on the internal/external VIP and the OpenStack +services. It also enables secure end-to-end communication between OpenStack +services that support TLS termination. The OpenStack services that support +backend TLS termination in Ussuri are: Keystone, Glance, Heat, Placement, +Horizon, Barbican, and Cinder. + +The configuration variables that control back-end TLS for service endpoints +are: + +- ``kolla_enable_tls_backend`` +- ``kolla_tls_backend_cert`` +- ``kolla_tls_backend_key`` +- ``haproxy_backend_cacert`` +- ``haproxy_backend_cacert_dir`` + +The default state for back-end TLS is disabled. To enable TLS for the back-end +communication: + +.. code-block:: yaml + + kolla_enable_tls_backend: "yes" + +It is also possible to enable back-end TLS on a per-service basis. For example, +to enable back-end TLS for Keystone, set ``keystone_enable_tls_backend`` to +``yes``. + +The default values for ``haproxy_backend_cacert`` and +``haproxy_backend_cacert_dir`` should suffice if the certificate is in the +system trust store. Otherwise, they should be configured to a location of the +CA certificate installed in the service containers. + +Each backend service requires a certificate and private key. In many cases it +is necessary to use a separate certificate and key for each host, or even +per-service. The following precedence is used for the certificate: + +* ``{{ kolla_certificates_dir }}/{{ inventory_hostname }}/{{ project_name + }}-cert.pem`` +* ``{{ kolla_certificates_dir }}/{{ inventory_hostname }}-cert.pem`` +* ``{{ kolla_certificates_dir }}/{{ project_name }}-cert.pem`` +* ``{{ kolla_tls_backend_cert }}`` + +And for the private key: + +* ``{{ kolla_certificates_dir }}/{{ inventory_hostname }}/{{ project_name + }}-key.pem`` +* ``{{ kolla_certificates_dir }}/{{ inventory_hostname }}-key.pem`` +* ``{{ kolla_certificates_dir }}/{{ project_name }}-key.pem`` +* ``{{ kolla_tls_backend_key }}`` + +The default for ``kolla_certificates_dir`` is ``/etc/kolla/certificates``. + +``kolla_tls_backend_cert`` and ``kolla_tls_backend_key``, default to ``{{ +kolla_certificates_dir }}/backend-cert.pem`` and ``{{ kolla_certificates_dir +}}/backend-key.pem`` respectively. + +``project_name`` is the name of the OpenStack service, e.g. ``keystone`` or +``cinder``. + +.. note:: + The back-end TLS cert/key can be the same certificate that is used for the + VIP, as long as those certificates are configured to allow requests from + both the VIP and internal networks. + +By default, the TLS certificate will be verified as trustable by the +OpenStack services. Although not recommended for production, it is possible to +disable verification of the backend certificate: + +.. code-block:: yaml + + kolla_verify_tls_backend: "no" + +.. _admin-tls-generating-a-private-ca: + +Generating a Private Certificate Authority +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. note:: + + The certificates generated by Kolla Ansible use a simple Certificate + Authority setup and are not suitable for a production deployment. Only + certificates signed by a trusted Certificate Authority should be used in a + production deployment. + +It's not always practical to get a certificate signed by a trusted CA. In a +development or internal test OpenStack deployment, it can be useful to +generate certificates locally to enable TLS. + +For convenience, the ``kolla-ansible`` command will generate the necessary +certificate files based on the information in the ``globals.yml`` +configuration file and the inventory file: + +.. code-block:: console + + kolla-ansible -i multinode certificates + +The ``certificates`` role performs the following actions: + +1. Generates a test root Certificate Authority +2. Generates the internal/external certificates which are signed by the root + CA. +3. If back-end TLS is enabled, generate the back-end certificate signed by + the root CA. + +The combined certificate and key file ``haproxy.pem`` (which is the default +value for ``kolla_external_fqdn_cert``) will be generated and stored in the +``/etc/kolla/certificates/`` directory, and a copy of the CA certificate +(``root.crt``) will be stored in the ``/etc/kolla/certificates/ca/`` directory.