From 3f2923671ce2822cce5dc978ad14d065c5cedf48 Mon Sep 17 00:00:00 2001
From: "James E. Blair" <jeblair@hp.com>
Date: Mon, 20 Oct 2014 15:44:32 -0700
Subject: [PATCH] Add AFS docs
Change-Id: I11a5e13a9aca71ea1368b6e9df61a606ea3746c3
---
doc/source/afs.rst | 192 ++++++++++++++++++++++++++++++++++++++++
doc/source/kerberos.rst | 19 +++-
doc/source/systems.rst | 1 +
3 files changed, 209 insertions(+), 3 deletions(-)
create mode 100644 doc/source/afs.rst
diff --git a/doc/source/afs.rst b/doc/source/afs.rst
new file mode 100644
index 0000000000..83578cb4a0
--- /dev/null
+++ b/doc/source/afs.rst
@@ -0,0 +1,192 @@
+:title: OpenAFS
+
+.. _openafs:
+
+OpenAFS
+#######
+
+The Andrew Filesystem (or AFS) is a global distributed filesystem.
+With a single mountpoint, clients can access any site on the Internet
+which is running AFS as if it were a local filesystem.
+
+OpenAFS is an open source implementation of the AFS services and
+utilities.
+
+A collection of AFS servers and volumes that are collectively
+administered within a site is called a ``cell``. The OpenStack
+project runs the ``openstack.org`` AFS cell, accessible at
+``/afs/openstack.org/``.
+
+At a Glance
+===========
+
+:Hosts:
+ * afsdb01.openstack.org (a vldb and pts server in DFW)
+ * afsdb02.openstack.org (a vldb and pts server in ORD)
+ * afs01.dfw.openstack.org (a fileserver in DFW)
+ * afs01.ord.openstack.org (a fileserver in ORD)
+:Puppet:
+ * :file:`modules/openafs`
+ * :file:`modules/openstack_project/manifests/afsdb.pp`
+ * :file:`modules/openstack_project/manifests/afsfs.pp`
+:Projects:
+ * http://openafs.org/
+:Bugs:
+ * http://bugs.launchpad.net/openstack-ci
+ * http://rt.central.org/rt/Search/Results.html?Order=ASC&DefaultQueue=10&Query=Queue%20%3D%20%27openafs-bugs%27%20AND%20%28Status%20%3D%20%27open%27%20OR%20Status%20%3D%20%27new%27%29&Rows=50&OrderBy=id&Page=1&Format=&user=guest&pass=guest
+:Resources:
+ * `OpenAFS Documentation <http://docs.openafs.org/index.html>`_
+
+OpenStack Cell
+--------------
+
+AFS may be one of the most thoroughly documented systems in the world.
+There is plenty of very good information about how AFS works and the
+commands to use it. This document will only cover the mininmum needed
+to understand our deployment of it.
+
+OpenStack runs an AFS cell called ``openstack.org``. There are three
+important services provided by a cell: the volume location database
+(VLDB), the protection database (PTS), and the file server (FS). The
+volume location service answers queries from clients about which
+fileservers should be contacted to access particular volumes, while
+the protection service provides information about users and groups.
+
+Our implementation follows the common recommendation to colocate the
+VLDB and PTS servers, and so they both run on our afsdb* servers.
+These servers all have the same information and communicate with each
+other to keep in sync and automatically provide high-availability
+service. For that reason, one of our DB servers is in the DFW region,
+and the other in ORD.
+
+Fileservers contain volumes, each of which is a portion of the file
+space provided by that cell. A volume appears as at least one
+directory, but may contain directories within the volume. Volumes are
+mounted within other volumes to construct the filesystem hierarchy of
+the cell.
+
+OpenStack has one fileserver in DFW and one in ORD. They do not
+automatically contain copies of the same data. A read-write volume in
+AFS can only exist on exactly one fileserver, and if that fileserver
+is out of service, the volumes it serves are not available. However,
+volumes may have read-write copies which are stored on other
+fileservers. If a client requests a read-only volume, as long as one
+site with a read-only volume is online, it will be available.
+
+Client Configuration
+--------------------
+
+To use OpenAFS on a Debian or Ubuntu machine::
+
+ sudo apt-get install openafs-client openafs-krb5 krb5-user
+
+Debconf will ask you for a default realm, cell and cache size.
+Answer::
+
+ Default Kerberos version 5 realm: OPENSTACK.ORG
+ AFS cell this workstation belongs to: openstack.org
+ Size of AFS cache in kB: 500000
+
+The default cache size in debconf is 50000 (50MB) which is not very
+large. We recommend setting it to 500000 (500MB -- add a zero to the
+default debconf value), or whatever is appropriate for your system.
+
+The OpenAFS client is not started by default, so you will need to
+run::
+
+ sudo service openafs-client start
+
+When it's done, you should be able to ``cd /afs/openstack.org``.
+
+Most of what is in our AFS cell does not require authentication.
+However, if you have a principal in kerberos, you can get an
+authentication token for use with AFS with::
+
+ kinit
+ aklog
+
+Administration
+--------------
+
+The following information is relevant to AFS administrators.
+
+All of these commands have excellent manpages which can be accessed
+with commands like ``man vos`` or ``man vos create``. They also
+provide short help messages when run like ``vos -help`` or ``vos
+create -help``.
+
+For all administrative commands, you may either run them from any AFS
+client machine while authenticated as an AFS admin, or locally without
+authentication on an AFS server machine by appending the `-localauth`
+flag to the end of the command.
+
+Adding a User
+~~~~~~~~~~~~~
+First, add a kerberos principal as described in :ref:`addprinc`. Have the
+username and UID from puppet ready.
+
+Then add the user to the protection database with::
+
+ pts createuser $USERNAME -id UID
+
+Admin UIDs start at 1 and increment. If you are adding a new admin
+user, you must run ``pts listentries``, find the highest UID for an
+admin user, increment it by one and use that as the UID. The username
+for an admin user should be in the form ``username.admin``.
+
+.. note::
+ Any '/' characters in a kerberos principal become '.' characters in
+ AFS.
+
+Adding a Superuser
+~~~~~~~~~~~~~~~~~~
+Run the following commands to add an existing principal to AFS as a
+superuser::
+
+ bos adduser -server afsdb01.openstack.org -user $USERNAME.admin
+ bos adduser -server afsdb02.openstack.org -user $USERNAME.admin
+ pts adduser -user $USERNAME.admin -group system:administrators
+
+Creating a Volume
+~~~~~~~~~~~~~~~~~
+
+Select a fileserver for the read-write copy of the volume according to
+which region you wish to locate it after ensuring it has sufficient
+free space. Then run::
+
+ vos create $FILESERVER a $VOLUMENAME
+
+The `a` in the preceding command tells it to place the volume on
+partition `vicepa`. Our fileservers only have one partition and therefore
+this is a constant.
+
+Be sure to mount the volume in AFS with::
+
+ fs mkmount /afs/openstack.org/path/to/mountpoint $VOLUMENAME
+
+You may want to create read-only sites for the volume with ``vos
+addsite`` and then ``vos release``.
+
+You should set the volume quota with ``fs setquota``.
+
+Adding a Fileserver
+~~~~~~~~~~~~~~~~~~~
+Put the machine's public IP on a single line in
+/var/lib/openafs/local/NetInfo (TODO: puppet this).
+
+Copy ``/etc/openafs/server/KeyFile`` from an existing fileserver.
+
+Create an LVM volume named ``vicepa`` from cinder volumes. See
+:ref:`cinder` for details on volume management. Then run::
+
+ mkdir /vicepa
+ echo "/dev/main/vicepa /vicepa ext4 errors=remount-ro,barrier=0 0 2" >>/etc/fstab
+ mount -a
+
+Finally, create the fileserver with::
+
+ bos create NEWSERVER dafs dafs \
+ -cmd "/usr/lib/openafs/dafileserver -p 23 -busyat 600 -rxpck 400 -s 1200 -l 1200 -cb 65535 -b 240 -vc 1200" \
+ -cmd /usr/lib/openafs/davolserver \
+ -cmd /usr/lib/openafs/salvageserver \
+ -cmd /usr/lib/openafs/dasalvager
diff --git a/doc/source/kerberos.rst b/doc/source/kerberos.rst
index 3a1a182996..b46e3f0ea1 100644
--- a/doc/source/kerberos.rst
+++ b/doc/source/kerberos.rst
@@ -54,10 +54,23 @@ Copy the file `/etc/krb5.keytab` to the second kdc host.
The puppet config sets up slave propogation scripts and cron jobs to run them.
+.. _addprinc:
+
Adding principals
-----------------
-To add an admin principal::
+First, ensure the user has an entry in puppet so they have a unix
+shell account on our hosts. SSH access is not necessary, but keeping
+track of usernames and uids with account entries is necessary.
- # execute kadmin.local then run these commands
- addprinc corvus/admin@OPENSTACK.ORG
+Then, add the user to Kerberos using kadmin (while authenticated as a
+kerberos admin) or kadmin.local on the kdc::
+
+ kadmin: addprinc $USERNAME@OPENSTACK.ORG
+
+Where `$USERNAME` is the lower-case username of their unix account in
+puppet. `OPENSTACK.ORG` should be capitalized.
+
+If you are adding an admin principal, use
+`username/admin@OPENSTACK.ORG`. Admins should additionally have
+regular user principals.
diff --git a/doc/source/systems.rst b/doc/source/systems.rst
index cecca35ee7..4121b0b2de 100644
--- a/doc/source/systems.rst
+++ b/doc/source/systems.rst
@@ -29,6 +29,7 @@ Major Systems
openstackid
storyboard
kerberos
+ afs
.. NOTE(dhellmann): These projects were not listed above, or in any
other toctree, which breaks the build. It's not clear why they were