c1aff2ed38
This adds a role and related testing to manage our Kerberos KDC servers, intended to replace the puppet modules currently performing this task. This role automates realm creation, initial setup, key material distribution and replica host configuration. None of this is intended to run on the production servers which are already setup with an active database, and the role should be effectively idempotent in production. Note that this does not yet switch the production servers into the new groups; this can be done in a separate step under controlled conditions and with related upgrades of the host OS to Focal. Change-Id: I60b40897486b29beafc76025790c501b5055313d
167 lines
5.8 KiB
ReStructuredText
167 lines
5.8 KiB
ReStructuredText
:title: Kerberos
|
|
|
|
.. _kerberos:
|
|
|
|
Kerberos
|
|
########
|
|
|
|
Kerberos is a computer network authentication protocol which works on the
|
|
basis of 'tickets' to allow nodes communicating over a non-secure network
|
|
to prove their identity to one another in a secure manner. It is the basis
|
|
for authentication to AFS.
|
|
|
|
At a Glance
|
|
===========
|
|
|
|
:Hosts:
|
|
* kdc*.openstack.org
|
|
:Ansible:
|
|
* :git_file:`playbooks/service-kerberos.yaml`
|
|
:Projects:
|
|
* http://web.mit.edu/kerberos
|
|
:Bugs:
|
|
* https://storyboard.openstack.org/#!/project/748
|
|
* http://krbdev.mit.edu/rt/
|
|
:Resources:
|
|
* `Kerberos Website <http://web.mit.edu/kerberos>`_
|
|
* `KDC Install guide <http://web.mit.edu/kerberos/krb5-devel/doc/admin/install_kdc.html>`_
|
|
|
|
OpenStack Realm
|
|
---------------
|
|
|
|
OpenStack runs a Kerberos ``Realm`` called ``OPENSTACK.ORG``. The
|
|
realm contains a ``Key Distribution Center`` or KDC which is spread
|
|
across a primary and a replica, as well as an admin server which only
|
|
runs on the primary.
|
|
|
|
Most of the configuration is in Ansible, but management of user
|
|
accounts, known as ``principals``, is a manual task for
|
|
administrators.
|
|
|
|
Realm Creation
|
|
--------------
|
|
|
|
Realm creation is exercised by the Ansible roles during testing, but
|
|
is not expected to be used in production (because we have an active
|
|
realm/database).
|
|
|
|
The general process is:
|
|
|
|
* create the new Kerberos database on the primary
|
|
* distribute the database ``stash`` file from the primary to
|
|
replicas, to allow them to unencrypt the database propogated to
|
|
them. This is created from a master key kept as a secret.
|
|
* create an admin user (password saved in file on primary server)
|
|
* add host principals for the primary and replica servers
|
|
* create keytabs on primary and replica servers (via the admin user),
|
|
which allows them to authenticate to each other.
|
|
* setup database propogation from primary to replicas with ``kprop``
|
|
(primary-side push) and ``kpropod`` (replica-side listen).
|
|
|
|
In a disaster recovery situation, we can provision a fresh realm and
|
|
recover principals from dump files (XXX: 2020-03-11 ianw -- dump file
|
|
backup to come).
|
|
|
|
.. _addprinc:
|
|
|
|
Adding A User Principal
|
|
-----------------------
|
|
|
|
First, ensure the user has an entry in Ansible so they have a Unix
|
|
shell account on our hosts. SSH access is not necessary, but keeping
|
|
track of usernames and uids with account entries is necessary.
|
|
|
|
If you are already an admin, you should authenicate with ``kinit
|
|
<username>/admin``. Otherwise you can use the ``kadmin.local`` tool
|
|
(instead of ``kadmin``) on the primary server, which by-passes
|
|
authenication and writes to the database directly.
|
|
|
|
Use ``kadmin`` to add the principal like so:
|
|
|
|
kadmin: addprinc $USERNAME@OPENSTACK.ORG
|
|
|
|
Where `$USERNAME` is the lower-case username of their unix account in
|
|
Ansible. `OPENSTACK.ORG` should be capitalized.
|
|
|
|
If you are adding an admin principal, use
|
|
`username/admin@OPENSTACK.ORG`. Admins should additionally have
|
|
regular user principals.
|
|
|
|
Adding A Service Principal
|
|
--------------------------
|
|
|
|
A service principal is one that corresponds to an application rather
|
|
than a person. There is no difference in their implementation, only
|
|
in conventions around how they are created and used. Service
|
|
principals are created without passwords and keytab files are used
|
|
instead for authentication. The program `k5start` can use keytab
|
|
files to automatically obtain Kerberos credentials (and AFS if
|
|
needed).
|
|
|
|
Add the service principal to Kerberos using kadmin (while
|
|
authenticated as a Kerberos admin) or kadmin.local on the kdc::
|
|
|
|
kadmin: addprinc -randkey service/$NAME@OPENSTACK.ORG
|
|
|
|
Where `$NAME` is the lower-case name of the service. `OPENSTACK.ORG`
|
|
should be capitalized.
|
|
|
|
Then save the principal's keytab::
|
|
|
|
kadmin: ktadd -k /path/to/$NAME.keytab service/$NAME@OPENSTACK.ORG
|
|
|
|
.. warning:: Each time ``ktadd`` is run, the key is rotated and
|
|
previous keytabs are invalidated.
|
|
|
|
These keytabs are then usually converted to base-64 and stored as
|
|
secret variables, and deployed to hosts via Ansible.
|
|
``mirror-update`` is probably a good example.
|
|
|
|
Resetting A User Principal's Password
|
|
-------------------------------------
|
|
|
|
If you've forgotten your user password (it happens!) then from a
|
|
shell on one of the KDCs, execute ``sudo kadmin.local`` and use the
|
|
``cpw $USERNAME@OPENSTACK.ORG`` command and enter your new password
|
|
twice as prompted. If you need to reset your admin principal, use
|
|
``cpw $USERNAME/admin@OPENSTACK.ORG`` instead.
|
|
|
|
No Service Outage Server Maintenance
|
|
------------------------------------
|
|
|
|
Should you need perform maintenance on the Kerberos server that requires
|
|
taking Kerberos processes offline you can do this by performing your
|
|
updates on a single server at a time.
|
|
|
|
`kdc03.openstack.org` is our primary server and `kdc04.openstack.org`
|
|
is the replica. Perform your maintenance on `kdc04.openstack.org`
|
|
first. Then once that is done we can prepare for taking down the
|
|
primary. On `kdc03.openstack.org` run::
|
|
|
|
root@kdc03:~# /usr/local/bin/run-kprop.sh
|
|
|
|
You should see::
|
|
|
|
Database propagation to kdc04.openstack.org: SUCCEEDED
|
|
|
|
Once this is done the replica is ready and we can take kdc03
|
|
offline. When kdc03 is back online rerun `run-kprop.sh` to ensure
|
|
everything is working again.
|
|
|
|
DNS Entries
|
|
-----------
|
|
|
|
Kerberos uses the following DNS entries::
|
|
|
|
_kpasswd._udp.openstack.org. 300 IN SRV 0 0 464 kdc03.openstack.org.
|
|
_kerberos-adm._tcp.openstack.org. 300 IN SRV 0 0 749 kdc03.openstack.org.
|
|
_kerberos-master._udp.openstack.org. 300 IN SRV 0 0 88 kdc03.openstack.org.
|
|
_kerberos._udp.openstack.org. 300 IN SRV 0 0 88 kdc03.openstack.org.
|
|
_kerberos._udp.openstack.org. 300 IN SRV 0 0 88 kdc04.openstack.org.
|
|
_kerberos.openstack.org. 300 IN TXT "OPENSTACK.ORG"
|
|
|
|
Be sure to update them if kdc servers change. We also maintain a
|
|
CNAME for convenience which points to the master kdc::
|
|
|
|
kdc.openstack.org. 300 IN CNAME kdc03.openstack.org.
|