Overview

We begin with a basic introduction of AFS and Kerberos. Next, we will configure an AFS cell using OpenAFS stable release 1.4.14 on RedHat Enterprise Linux 6. The steps, however, should work with little or no difference on other Linux distributions and other releases of OpenAFS. We will first configure Kerberos 5 services and then OpenAFS.

Back to top

Brief introduction to AFS

IBM AFS, or OpenAFS, is a network-based distributed file system on client-server architecture where cooperating hosts share their file system resources to be used by clients both across local and wide area networks. It was originally developed at the Information Technology Center at Carnegie-Mellon University in Pittsburgh, Pennsylvania. The main strengths of AFS are:

• Global namespace
• Location transparency
• Caching facility
• Scalability
• Security features, and
• Robust communication protocol

These features allow AFS to be known as the granddaddy of distributed file systems even after so many years of development. AFS is available on all the major platforms in use today.

Architecturally, AFS has two components, a server and a client. Following is a brief description of AFS architecture.

• Server component (Vice) —AFS servers are monitored by a super-server called a Basic OverSeer Server (BOS Server). It monitors the health of AFS processes on the machine and may be used to start, stop, suspend or resume these processes, or to update new server binaries. The rest of the servers fall into two categories, namely database and file.
• Database servers — The following servers belong to the database servers category:
• Backup server (buserver) — Maintains backup database to administer all backup operations on the database servers.
• Volume location server (vlserver) — Maintains volume location database (VLDB) to track all information about volumes and their location on a file server. AFS volume is explained below. Vlserver supports operations such as queries of volume location and status, volume ID management, creation, deletion and modification of VLDB entries.
• Protection server (ptserver) — Maintains protection database (PDB) that maps between user or group names and their internal numerical AFS identifiers. It is also used to create, destroy, modify, and query ownership and membership.
• Authentication server (kaserver) — Maintains authentication database that it uses to authenticate users and responds successful clients with authentication tickets.
• File servers
• File server (fileserver) — Handles files and directories residing in AFS volumes, giving access to a properly authenticated client request. It provides flat file service.
• Salvager server (salvager) — Provides best effort to recover any data stored in AFS volumes on a file server.
• Volume server (volser) — Allows administrative tasks related to AFS volumes such as creation, relocation, deletion, replication, etc.
• Client component (Venus) — AFS cache management is handled by an AFS client (afsd), with the help of a set of daemons. It supports both local disk caching as well as caching in memory.

AFS servers are user-level applications, however, AFS has a kernel module that intercepts calls to AFS space.

Back to top

Basic AFS concepts

A set of server and client machines managed and operated by an administratively independent organization is called an AFS cell. In an AFS cell, at least one of the machines must be running database server and file server processes. The rest of the machines may just run the client component.

AFS stores its files in volumes which uses storage space on machines running the file server component of AFS. An AFS volume may be considered to be a container holding files and directories. These volumes are designed so system administrators may move these volume for load balancing, etc., from one machine running file server process to another machine running file server process, within the same AFS cell. There are a set of operations which may be performed on these AFS volumes such as replication, dumping of data, backup, restoration, etc. There are many other concepts related to volumes which are beyond the scope of this article.

Database server processes running on different machines in an AFS cell use Ubik protocol to keep their databases in sync. Ubik protocol helps to replicate databases such as VLDB, ADB and PDB consistently across multiple server machines to improve their availability and share access loads among replication sites. It uses a quorum completion algorithm to synchronize database information.

Another important concept is the communication mechanism used by an AFS server as well as client processes within or across AFS cells which is called Rx. It is a Remote Procedure Call (RPC) mechanism providing a high-performance, multi-threaded and secure mechanism for communication between processes running on the same machine, on a Local Area Network (LAN) as well as Wide Area Network(WAN). The Rx library is suitably designed to be used in kernel as well as user space applications with similar API interfaces.

Back to top

Brief introduction to Kerberos

Kerberos is a network authentication protocol, developed at MIT, which uses a shared symmetric key to establish mutual authentication. Similar cryptographic keys can be used to encrypt as well as decrypt messages, therefore, both the sides are guaranteed that they are interacting with the correct one.

Basic Kerberos concepts

This section defines the basic Kerberos concepts used in this article. Next, we will configure a Kerberos realm. A realm represents an authentication administrative domain. It bounds the authority of an authentication aerver or AS (a component of Kerberos Infrastructure) of that realm for providing authentication to the principals who belong to the realm. Kerberos principals are entities such as users, machines or services which need authentication service. Principals share their cryptographic key with the authentication server of the realm to which they belong.

Two important concepts pertaining to a principal are keys and tickets.

• Key — A key is cryptographic data which is usually derived from a user's password using specific encryption algorithms. For principals other than users, a key is generated randomly. A Key Version Number (kvno) is used to tag keys.
• Ticket — A ticket is authentication data which a client presents for authenticating itself to a server for a service. A ticket is provided by an authentication server to the client using a cryptographic key of the intended server. Therefore, a client is unable peek or poke the ticket content.

In a Kerberos infrastructure, we have something called a key distribution center, or KDC, which provides ticket distributing functionality. Logically it provides three services, principals database maintenance, authentication server (AS), and ticket granting server (TGS). KDC runs two important daemons:

• kadmind
  This is an administrative daemon managing a database of principals, their secret keys, and policy configurations. kadmin.local and kadmin commands allow access to this functionality on local machines or remote machines running KDC, respectively.
• krb5kdc
  This is the significant daemon dealing with authentication requests. When a users requests an authentication ticket, it comes to this daemon.
  

Back to top

Kerberos 5 configuration

Before continuing, we need to be sure binaries related to Kerberos 5 are installed on the machine. Following are the RPMs that must be installed for compiling and using OpenAFS with Kerberos 5 support.

1. krb5-workstation
2. krb5-server
3. krb5-libs
4. krb5-devel

Listing 1 shows the RPMs installed on my test machine. It has additional RPMs installed apart from those mentioned above. The listing also shows a brief description of each.

root@rhel6vm1 ~ # rpm -qa | grep krb5
krb5-workstation-1.8.2-3.el6.i686  <-- Basic Kerberos 5 related programs like kinit etc
krb5-auth-dialog-0.13-3.el6.i686   <-- For getting dialog box for Krb5 Tickets
krb5-appl-servers-1.0.1-1.el6.i686 <-- Kerberized application servers
krb5-server-1.8.2-3.el6.i686  <-- KDC and related programs for Kerberos 5
krb5-libs-1.8.2-3.el6.i686  <-- Shared libraries used by Kerberos 5
krb5-pkinit-openssl-1.8.2-3.el6.i686 <-- PKINIT plugin for Kerberos 5
krb5-appl-clients-1.0.1-1.el6.i686   <-- Kerberized clients for application servers
pam_krb5-2.3.11-1.el6.i686   <-- Pluggable Authentication Module for Kerberos 5
krb5-devel-1.8.2-3.el6.i686 <-- For developing Kerberos 5 based applications
root@rhel6vm1 ~ #

We are going to set up a Kerberos realm named VMKRB5. The hostname of the machine we are using is rhel6vm1.in.ibm.com. The main configuration file for Kerberos 5 is /etc/krb5.conf. This configuration file is comprised of sections for specifying logging options, admin server and KDC server instance information for a particular realm, and values of options for Kerberos, etc.

Provided next is information about KDC and Admin Server for Kerberos realm VMKRB5, in realm section. The information about other realms can be specified in the same section. Realms are specified for different domains in the domain_realm section. For more information on this file, see the manual pages of krb5.conf.

Listing 2 provides a sample of /etc/krb5.conf. You will need to modify:

• default_realm value
• domain_realm value
• values of kdc, and
• admin_server

for your Kerberos realm.

[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 default_realm = VMKRB5
 dns_lookup_realm = false
 dns_lookup_kdc = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 allow_weak_crypto = true

[realms]

VMKRB5 = {
  kdc = rhel6vm1.in.ibm.com:88
  admin_server = rhel6vm1.in.ibm.com:749
 }

[domain_realm]
 .in.ibm.com = VMKRB5
 in.ibm.com = VMKRB5

[kdc]
 profile = /var/kerberos/krb5kdc/kdc.conf

[appdefaults]

 pam = {
   debug = false
   ticket_lifetime = 36000
   renew_lifetime = 36000
   forwardable = true
   krb4_convert = false
 }

The behavior of KDC server is governed by configuration file /var/kerberos/krb5kdc/kdc.conf. We will specify the ACL (Access Control List) file, encryption types supported by KDC, etc., among other options for Kerberos realms needed by KDC. In Listing 3, a sample configuration file is shown for KDC. The realm name needs to be changed appropriately in the realms section as per their configuration.

[kdcdefaults]
 v4_mode = nopreauth
 kdc_tcp_ports = 88

[realms]
VMKRB5 = {
  master_key_type = des-cbc-crc
  acl_file = /var/kerberos/krb5kdc/kadm5.acl
  dict_file = /usr/share/dict/words
  admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
  supported_enctypes = des-cbc-crc:normal des-cbc-crc:v4 des-cbc-crc:afs3
 }

Please note KDC is configured to support encryption types which are a variant of des-cbc-crc, as it works well with OpenAFS. Kerberos 5 supports many other encryption types as well. For more information on the configuration file kdc.conf, see the manual pages of kdc.conf.

Listing 4 is a sample configuration file defining ACLs. Administrators may chose to define ACLs as per their requirements. The access control list file below allows all principals with admin instance for our realm VMKRB5 all accesses.

*/admin@VMKRB5     *

Initializing Kerberos 5 database

We will first initialize the database for Kerberos 5 for realm VMKRB5 using Kerberos Database Maintenance Utility kdb5_util. The initialization of Kerberos Database using kdb5_util is shown in Listing 5.

root@rhel6vm1 ~ # kdb5_util create -r VMKRB5 -s
Loading random data
Initializing database '/var/kerberos/krb5kdc/principal' for realm 'VMKRB5',
master key name 'K/M@VMKRB5.' You will be prompted for the database Master Password.
    It is important that you NOT FORGET this password.
Enter KDC database master key: <vmkrb5>
Re-enter KDC database master key to verify: <vmkrb5>

The -s option tells kdb5_util to create a stash file (a file containing a set of secret keys) to authenticate itself. If there are multiple realms defined in /etc/krb5.conf, you should use option -r to specify the realm for which you are going to create the database.

Starting Kerberos 5 services

Listing 6 shows commands to start Kerberos 5 services. You may use the chkconfig tool to schedule this service to run at specific run-levels.

root@rhel6vm1 ~ # service kadmin start
Starting Kerberos 5 Admin Server:                          [  OK  ]
root@rhel6vm1 ~ # service krb5kdc start
Starting Kerberos 5 KDC:                                   [  OK  ]

Defining basic policies and user accounts

kadmin.local command suite can be used on machines running KDC. listprincs subcommand is an alias for list_principals which lists existing principals in the database.

Subcommand add_policy or addpol allows us to define policies. Here we are defining policies named admin and service. These policies mandate the minimum length of password and minimum number of classes (as in alphabets, numerals, and special characters, etc).

listpol lists all the defined policies. getpol gives detailed information on the specified policy.

Listing 7 shows basic usage of these commands.

root@rhel6vm1 ~ # kadmin.local
Authenticating as principal avinesh/admin@VMKRB5 with password.
kadmin.local:  listprincs
K/M@VMKRB5 <-- Kerberos Master Key Principal
kadmin/admin@VMKRB5
kadmin/changepw@VMKRB5
kadmin/history@VMKRB5
kadmin/rhel6vm1.in.ibm.com@VMKRB5
krbtgt/VMKRB5@VMKRB5
kadmin.local:  add_policy -minlength 8 -minclasses 3 admin
kadmin.local:  addpol -minlength 8 -minclasses 4 service
kadmin.local:  listpols
admin
service
kadmin.local:  getpol admin
Policy: admin
Maximum password life: 0
Minimum password life: 0
Minimum password length: 8
Minimum number of password character classes: 3
Number of old keys kept: 1
Reference count: 0
Maximum password failures before lockout: 0
Password failure count reset interval: 0
Password lockout duration: 0

All kadmin/* principals, as shown in above listing as output of listprocs, are for varying administration tasks. krbtgt/VMKRB5@VMKRB5 is a principal for the Ticket Granting Ticket Service.

We are also creating a principal with the name root/admin for our realm VMKRB5, which complies to the admin policy we just defined. We will use this principal with kadmin, as shown in Listing 8., for administrative needs from any machine in the realm.

kadmin.local:  addprinc -policy admin root/admin
Enter password for principal "root/admin@VMKRB5": <r00t123$>
Re-enter password for principal "root/admin@VMKRB5": <r00t123$>
Principal "root/admin@VMKRB5" created.
kadmin.local:  q

Creation of AFS principal and service keys

The snippet below in Listing 9. shows the use of kadmin command suite to create principals afs/krb5afs.in.ibm.com, admin, and avinesh. Ktadd adds the key for the specified principal in the given Kerberos Key Table (keytab) file. We should make sure that the keytab file does not exist before we issue ktadd as we want only one key entry for afs/krb5afs.in.ibm.com. The Key Version Number (kvno) 2 is of particular importance here because we need to specify the same kvno while importing the key in the KeyFile to be used by AFS.

root@rhel6vm1 ~ # rm -f /etc/krb5.keytab.afs
root@rhel6vm1 ~ # kadmin -p root/admin
Authenticating as principal root/admin with password.
Password for root/admin@VMKRB5:
kadmin:  addprinc -policy service -randkey -e des-cbc-crc:normal afs/krb5afs.in.ibm.com
Principal "afs/krb5afs.in.ibm.com@VMKRB5" created.
kadmin:  ktadd -e des-cbc-crc:normal -k /etc/krb5.keytab.afs afs/krb5afs.in.ibm.com
Entry for principal afs/krb5afs.in.ibm.com with kvno 2, encryption type DES cbc mode with 
CRC-32 added to keytab WRFILE:/etc/krb5.keytab.afs.
kadmin:  addprinc -policy admin admin
Enter password for principal "admin@VMKRB5": <admin123$>
Re-enter password for principal "admin@VMKRB5": <admin123$>
Principal "admin@VMKRB5" created.
kadmin:  addprinc -policy admin avinesh
Enter password for principal "avinesh@VMKRB5": <avinesh123$>
Re-enter password for principal "avinesh@VMKRB5": <avinesh123$>
Principal "avinesh@VMKRB5" created.
kadmin:  q

klist reads the keytab file and gives detailed information about all the keys in the file.

root@rhel6vm1 ~ # klist -kte /etc/krb5.keytab.afs
Keytab name: WRFILE:/etc/krb5.keytab.afs
KVNO Timestamp         Principal
---- ----------------- --------------------------------------------------------
   2 01/26/11 23:14:27 afs/krb5afs.in.ibm.com@VMKRB5 (DES cbc mode with CRC-32)

Back to top

Getting OpenAFS source and doing a build of OpenAFS

We need to download OpenAFS source from their website using wget, the tool for downloads on Linux machines, to download a compressed tar file of OpenAFS source.

root@rhel6vm1 ~ # wget http://openafs.org/dl/openafs/1.4.14/openafs-1.4.14-src.tar.bz2
--2011-01-17 12:22:29-- http://openafs.org/dl/openafs/1.4.14/openafs-1.4.14-src.tar.bz2
Resolving openafs.org... 128.2.200.90
Connecting to openafs.org|128.2.200.90|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://dl.openafs.org/dl/openafs/1.4.14/openafs-1.4.14-src.tar.bz2 [following]
--2011-01-17 12:22:30-- http://dl.openafs.org/dl/openafs/1.4.14/openafs-1.4.14-src.tar.bz2
Resolving dl.openafs.org... 128.2.201.26
Connecting to dl.openafs.org|128.2.201.26|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11813993 (11M) [application/x-tar]
Saving to: `openafs-1.4.14-src.tar.bz2'

82% [========================================>         ] 9,709,310   22.4K/s  eta 57s

Configuring OpenAFS source for doing a build

It is worth reading the output of ./configure --help to see the configuration options available with OpenAFS.

The following listing shows commands to configure OpenAFS source with Kerberos 5 support. make is for the build and make dest creates a Transarc style directory structure with binaries.

root@rhel6vm1 openafs-1.4.14 # which krb5-config
/usr/bin/krb5-config
root@rhel6vm1 openafs-1.4.14 # ./configure --with-afs-sysname=i386_linux26 
                                --enable-transarc-paths \
                                --with-krb5-conf=/usr/bin/krb5-config
root@rhel6vm1 openafs-1.4.14 # make
root@rhel6vm1 openafs-1.4.14 # make dest

Creation of AFS specific directories

OpenAFS supports disk-based as well as memory-based caching. When an OpenAFS client is configured to do local disk caching, it conventionally stores caching-related data in a directory labeled /usr/vice/cache. It is recommended to use a separate partition for storing the cached data. Typically, a user would create a partition, make a file system (usually ext3 in Linux), and mount the file system on the directory /usr/vice/cache. If AFS administrators choose to use a different directory for this, they should specify the same in /usr/vice/etc/cacheinfo file. More about /usr/vice/etc/cacheinfo is explained in section AFS client disk cache configuration below.

A machine running AFS File Server needs to have at least one partition or logical volume for storing AFS Volumes. Each such partition is mounted at a directory named /vicepxx, with xx meaning one or two alphabetical lower case letters. Administrators usually have partitions as /vicepa through /vicepz, and then /vicepaa through /vicepaz. If more vice partitions are needed, /vicepba through /vicepbz are used, and so on. OpenAFS allows users to use regular directory /vicepxx to store AFS volumes if it contains a file named AlwaysAttach. It is advised, however, to have separate partitions for an AFS server instead of a regular directory.

root@rhel6vm1 openafs-1.4.14 # mkdir /afs
root@rhel6vm1 openafs-1.4.14 # chmod 777 /afs
root@rhel6vm1 openafs-1.4.14 # mkdir -p /usr/vice/etc
root@rhel6vm1 openafs-1.4.14 # mkdir -p /usr/afs
root@rhel6vm1 openafs-1.4.14 # mkdir -p /usr/afsws

root@rhel6vm1 openafs-1.4.14 # cd i386_linux26/dest
root@rhel6vm1 dest # cp -rp root.client/usr/vice/etc/* /usr/vice/etc
root@rhel6vm1 dest # cp -p  root.client/usr/vice/etc/afs.rc /etc/rc.d/init.d/afs
root@rhel6vm1 dest # cp root.client/usr/vice/etc/afs.conf /etc/sysconfig/afs
root@rhel6vm1 dest # cp -rp root.server/usr/afs/* /usr/afs
root@rhel6vm1 dest # cp -rp bin/ etc/ include/ lib/ /usr/afsws
root@rhel6vm1 dest # ln -s /usr/afs/bin/bos /usr/bin/bos

The configuration file /etc/sysconfig/afs has options that need to be tweaked for running AFS client, server, and cache manager's parameters (like caching type - on disk or in memory, and other important variables). If both AFS client and AFS server will be running, the file /etc/sysconfig/afs must have both AFS_CLIENT as well as AFS_SERVER set to "on."

Back to top

Configuration of AFS cell

Here we are going to configure an AFS cell with cell name krb5afs.in.ibm.com on our machine rhel6vm1.in.ibm.com.

For convenience during OpenAFS installation, we will store the cell name and the machine name (which will run file servers) to environment variables CELLNAME and SERVERNAME, respectively.

root@rhel6vm1 ~ # export CELLNAME=krb5afs.in.ibm.com
root@rhel6vm1 ~ # export SERVERNAME=rhel6vm1.in.ibm.com

Setting AFS cellname

As shown in Listing 14, we start bosserver in noauth mode so that it does not need us to be authorized.

root@rhel6vm1 ~ # /usr/afs/bin/bosserver -noauth &
[1] 25631
root@rhel6vm1 ~ #
[1]+  Done                    /usr/afs/bin/bosserver -noauth
root@rhel6vm1 ~ # /usr/afs/bin/bos setcellname $SERVERNAME $CELLNAME -noauth
root@rhel6vm1 ~ # /usr/afs/bin/bos listhosts $SERVERNAME -noauth
Cell name is krb5afs.in.ibm.com
    Host 1 is rhel6vm1.in.ibm.com

bos setcellname allows the setting of a cellname and bos listhosts gives information of server machines in the AFS cell.

The code snippet in Listing 15 shows creation of AFS protection server (ptserver) and AFS volume location server (vlserver) instances.

Back to top

AFS server configuration

root@rhel6vm1 ~ # /usr/afs/bin/bos create $SERVERNAME ptserver simple \
                               /usr/afs/bin/ptserver -cell $CELLNAME -noauth
root@rhel6vm1 ~ # /usr/afs/bin/bos create $SERVERNAME vlserver simple \
                               /usr/afs/bin/vlserver -cell $CELLNAME -noauth

Adding keys to be used by AFS

We will use asetkey command to add the keys we exported earlier in keytab file /etc/krb5.keytab.afs for AFS Service Principal afs/krb5afs.in.ibm.com so that it is usable by AFS. Note that we specify the same kvno value of 2 here as we had mentioned above.

root@rhel6vm1 ~ # /usr/afs/bin/asetkey add 2 /etc/krb5.keytab.afs afs/krb5afs.in.ibm.com
root@rhel6vm1 ~ # /usr/afs/bin/bos listkeys $SERVERNAME -cell $CELLNAME
key 2 has cksum 4052960223
Keys last changed on Fri Jan 21 12:33:42 2011.
All done.

bos listkeys displays AFS server encryption key information stored in file /usr/afs/etc/KeyFile.

Adding privileged users

The commands in Listing 17 defines a privileged user who would be authorized to do administrative work for this AFS cell. The information about these administrative users are stored in file /usr/afs/etc/UserList on the database server machine.

root@rhel6vm1 ~ # /usr/afs/bin/bos adduser $SERVERNAME admin \
                                             -cell $CELLNAME -noauth
root@rhel6vm1 ~ # /usr/afs/bin/bos adduser $SERVERNAME avinesh \
                                             -cell $CELLNAME -noauth
root@rhel6vm1 ~ # /usr/afs/bin/bos listusers -s $SERVERNAME \
                                             -cell $CELLNAME -noauth
SUsers are: admin avinesh

bos adduser adds users admin and avinesh to the list of privileged users. bos listusers lists all such users from the UserList file.

Adding users to AFS protection server's database

Listing 18 shows creation of users admin and avinesh using the createuser option of the pts tool. We add these users to the predefined group of AFS system administrators system:administrators using option adduser of pts.

root@rhel6vm1 ~ # /usr/afs/bin/pts createuser -name admin   -noauth
User admin has id 1
root@rhel6vm1 ~ # /usr/afs/bin/pts createuser -name avinesh   -noauth
User avinesh has id 2
root@rhel6vm1 ~ # /usr/afs/bin/pts adduser admin system:administrators -noauth
root@rhel6vm1 ~ # /usr/afs/bin/pts adduser avinesh system:administrators -noauth
root@rhel6vm1 ~ # /usr/afs/bin/pts membership admin -noauth
Groups admin (id: 1) is a member of:
  system:administrators
root@rhel6vm1 ~ # /usr/afs/bin/pts membership avinesh -noauth
Groups avinesh (id: 2) is a member of:
  system:administrators
root@rhel6vm1 ~ # /usr/afs/bin/pts listentries -c $CELLNAME -localauth
Name                          ID  Owner Creator
anonymous                  32766   -204    -204
admin                          1   -204   32766
avinesh                        2   -204   32766

The membership option shows the membership details of a particular user. pts listentries lists down all the users in the protection server's database along with their ID.

Starting AFS File Server processes

Listing 19 shows commands to create and start file server processes. We will use bos status command to verify that these processes are running.

root@rhel6vm1 ~ # /usr/afs/bin/bos restart $SERVERNAME -all -cell $CELLNAME -noauth
root@rhel6vm1 ~ # /usr/afs/bin/bos create $SERVERNAME fs fs \
                                        /usr/afs/bin/fileserver \
                                        /usr/afs/bin/volserver  \
                                        /usr/afs/bin/salvager   \
                                        -cell $CELLNAME -noauth
root@rhel6vm1 ~ # /usr/afs/bin/bos status $SERVERNAME fs -long -noauth
Instance fs, (type is fs) currently running normally.
    Auxiliary status is: file server running.
    Process last started at Fri Jan 21 10:45:11 2011 (2 proc starts)
    Command 1 is '/usr/afs/bin/fileserver'
    Command 2 is '/usr/afs/bin/volserver'
    Command 3 is '/usr/afs/bin/salvager'

Creation of AFS volumes

A single volume is chosen as root of AFS file space for a given organization. By convention, this volume is named as root.afs. Each client in this organization performs a UNIX mount() of this root volume on /afs, thus attaching the entire AFS namespace at this point.

AFS Cache manager mounts root of AFS file system usually at /afs or as defined in configuration file /usr/vice/etc/cacheinfo. AFS Cache manager internally mounts root.afs volume to path /afs. The volume root.afs contains mount points of root volumes of all cells as described in the database of all cells known to this client, in file /usr/vice/etc/CellServDB. If a client needs to access a foreign cell (i.e., a cell other than a local one), we need to provide server details of that cell in this file.

root@rhel6vm1 ~ # /usr/afs/bin/vos create -s $SERVERNAME -pa a -name root.cell \
                                                    -maxq 30000 -c $CELLNAME -noauth
Volume 536870912 created on partition /vicepa of rhel6vm1.in.ibm.com
root@rhel6vm1 ~ # /usr/afs/bin/vos create -s $SERVERNAME -pa a -name root.afs \
                                                   -maxq 30000 -c $CELLNAME -noauth
Volume 536870915 created on partition /vicepa of rhel6vm1.in.ibm.com
root@rhel6vm1 ~ # /usr/afs/bin/vos listvol -s $SERVERNAME -c $CELLNAME -noauth
Total number of volumes on server rhel6vm1.in.ibm.com partition /vicepa: 2
root.afs                          536870915 RW          2 K On-line
root.cell                         536870912 RW          2 K On-line

Total volumes onLine 2 ; Total volumes offLine 0 ; Total busy 0

Back to top

AFS client configuration

Configuring AFS client and server to a particular cell

By default, the installation process creates configuration file /usr/vice/etc/ThisCell as a symbolic link to /usr/afs/etc/ThisCell, with the cell name we just chose as its content. The file /usr/vice/etc/ThisCell contains local cell names for this AFS client whereas /usr/afs/etc/ThisCell contains cell names whose database servers will be running on this machine. A machine can run AFS client and AFS servers for the same cell as well as different cells. If you need to AFS client to one AFS cell with AFS servers running for a different AFS cell, then we need to have /usr/vice/etc/ThisCell as a separate file instead of a symbolic link, with the desired cell names as content.

With the commands shown in Listing 21, we are configuring the machine to run AFS client as well AFS server for the same cell, however, we will have both the configuration file /usr/afs/etc/ThisCell and /usr/vice/etc/ThisCell distinct.

root@rhel6vm1 ~ # ls -l /usr/vice/etc/ThisCell
lrwxrwxrwx. 1 root root 21 Jan 21 10:31 /usr/vice/etc/ThisCell -> /usr/afs/etc/ThisCell
root@rhel6vm1 ~ # cat /usr/afs/etc/ThisCell
krb5afs.in.ibm.com
root@rhel6vm1 ~ # rm -f /usr/vice/etc/ThisCell
root@rhel6vm1 ~ # cp /usr/afs/etc/ThisCell  /usr/vice/etc/ThisCell
root@rhel6vm1 ~ # cat /usr/vice/etc/ThisCell
krb5afs.in.ibm.comroot@rhel6vm1 ~ # cat /usr/afs/etc/ThisCell
krb5afs.in.ibm.com

Access to foreign AFS cells

The configuration files /usr/vice/etc/CellServDB and /usr/afs/etc/CellServDB contain database server information for AFS client and AFS server respectively. The /usr/afs/etc/CellServDB file contains database server information of AFS cell for which a particular machine is running AFS server. It has information about one AFS cell. The file /usr/vice/etc/CellServDB, however, contains details of database servers of all AFS cell that this AFS client needs to access.

Again, the installation process creates /usr/vice/etc/CellServDB as a symbolic link to /usr/afs/etc/CellServDB. Following we will keep these two files separate in cases where a client needs to access other AFS cells. Simply add the cell information to /usr/vice/etc/CellServDB file.

root@rhel6vm1 ~ # cat /usr/afs/etc/CellServDB
>krb5afs.in.ibm.com     #Krb5 AFS Test Cell
192.168.110.129    #rhel6vm1.in.ibm.com
root@rhel6vm1 ~ # ls -l /usr/vice/etc/CellServDB
lrwxrwxrwx 1 root root 23 Jan 21 10:31 /usr/vice/etc/CellServDB -> /usr/afs/etc/CellServDB
root@rhel6vm1 ~ # rm -f /usr/vice/etc/CellServDB
root@rhel6vm1 ~ # cp /usr/afs/etc/CellServDB  /usr/vice/etc/CellServDB

AFS client disk cache configuration

AFS can be configured to cache data locally on disk or in memory. This can be specified in AFS configuration file /etc/sysconfig/afs. The configuration file /usr/vice/etc/cacheinfo has information about mount point for AFS File System, path where AFS will store data (when doing disk caching), and the number of blocks to use for cache size. Here we are specifying cache of size 8,15,000 KB:

root@rhel6vm1 ~ # cat /usr/vice/etc/cacheinfo
/afs:/usr/vice/cache:815000

Back to top

Final step

At this point we are almost done with configuration. We still need to stop AFS services running currently using command service afs stop. AFS Cache Manager mounts the volume root.afs at /afs (or as mentioned in /usr/vice/etc/cacheinfo file). Now we will be mounting the root.cell volume we created earlier.

The convention that is followed is to mount it on a directory named with the cell name under /afs. Therefore, we will be mounting root.cell volume to /afs/krb5afs.in.ibm.com. Its very important to disable the dynroot option to AFS client (afsd). To do this, we need to modify the /etc/sysconfig/afs file and set ENABLE_DYNROOT=off. With dynroot, the /afs directory is populated with sub-directories with AFS cell names as defined in /usr/vice/etc/CellServDB file. It does not load the volume root.afs from the file server and it goes after the root.cell volume directly, and only when a user tries to access the directory under which the volume root.cell is mounted.

We also need to have the realm name in file /usr/afs/etc/krb.conf as our realm name is different from the AFS cell name. Listing 23 shows commands to have the realm name in /usr/afs/etc/krb.conf, starting AFS services, setting ACL on /afs and mounting of root.cell volume.

root@rhel6vm1 ~ # service afs stop
Stopping AFS services.....
Stopping AFS bosserver
root@rhel6vm1 ~ #
root@rhel6vm1 / # echo "VMKRB5" > /usr/afs/etc/krb.conf
root@rhel6vm1 ~ # service afs start
Starting AFS services.....
afsd: All AFS daemons started.
root@rhel6vm1 / # /usr/afs/bin/fs setacl /afs system:anyuser rl
root@rhel6vm1 / # cd /afs
root@rhel6vm1 /afs # /usr/afs/bin/fs mkm krb5afs.in.ibm.com -vol root.cell \
                                                       -c krb5afs.in.ibm.com
root@rhel6vm1 /afs # ln -s krb5afs.in.ibm.com krb5afs

Back to top

All done!

We are now done with configuration of AFS as well as Kerberos 5. It is good to restart the machine after doing these configurations. Once the machine comes up, we will start Kerberos 5 and AFS services. At this point, we may enable the dynroot option for afsd, before starting AFS services.

In Listing 24, we use kinit to get a Kerberos 5 ticket for principal avinesh. klist shows a listing of all cached Kerberos 5 tickets.

root@rhel6vm1 ~ # kinit avinesh
Password for avinesh@VMKRB5:
root@rhel6vm1 ~ # klist -5f
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: avinesh@VMKRB5

Valid starting     Expires            Service principal
01/26/11 23:18:35  01/27/11 23:18:35  krbtgt/VMKRB5@VMKRB5
        renew until 01/26/11 23:18:35, Flags: FRI

aklog is used to set AFS tokens based on Kerberos 5 tickets. Option -d is optional as it shows verbose information on its working. tokens shows AFS tokens held by the AFS Cache Manager, as shown in Listing 25. When an AFS client tries to access AFS space, it may need to provide the token data to obtain access to the data it is requesting.

		
root@rhel6vm1 ~ # service afs start
Starting AFS services.....
afsd: All AFS daemons started.
root@rhel6vm1 ~ #
root@rhel6vm1 ~ # aklog -d
Authenticating to cell krb5afs.in.ibm.com (server rhel6vm1.in.ibm.com).
Trying to authenticate to user's realm VMKRB5.
Getting tickets: afs/krb5afs.in.ibm.com@VMKRB5
Using Kerberos V5 ticket natively
About to resolve name avinesh to id in cell krb5afs.in.ibm.com.
Id 2
Set username to AFS ID 2
Setting tokens. AFS ID 2 /  @ VMKRB5
root@rhel6vm1 ~ #
root@rhel6vm1 ~ # tokens

Tokens held by the Cache Manager (UID Based Tokens):

User's (AFS ID 1) tokens for afs@krb5afs1.in.ibm.com [Expires Feb  8 07:00]
   --End of list--
root@rhel6vm1 ~ #

Back to top

Summary

This article explains how to enter the vast world of AFS using OpenAFS and Kerberos 5 KDC for authentication. There are many more features in AFS which have been available to System Administrators for over 20 years. For more information on AFS and Kerberos 5, please review the Resources section below and follow the links to that documentation.

Back to top

Acknowledgment

I am thankful to Anthony Todd DeSantis of IBM AFS Support and Development for taking time out from his busy schedule to give useful comments and inputs to improve this article.

Resources

Learn

• OpenAFS.org: Learn more about the outsourced version of IBM AFS.
  
  
• "Documentation of OpenAFS", provides documentation for users as well as administrators.
  
  
• "For understanding Kerberos 5 Protocol", is a very good article on MIT Kerberos Consortium from Fulvio Ricciardi explaining Kerberos 5 protocol details
  
  
• "Easy migration to OpenAFS with Kerberos-5 support", a developerWorks article, provides comparative view between Kaserver based setup vs. Kerberos 5 based setup.
  
  
• "Resources on the IBM Network Authentication Service and related technologies for AIX", is one of many articles related to IBM Network Authentication Service by IBM developerWorks Master Author, Sandeep Ramesh Patil.
  
  
• developerWorks podcasts: Tune into interesting interviews and discussions for software developers
  
  
• Technical events and webcasts: Stay current with developerWorks Live! briefings.
  
  
• developerWorks on Twitter: Follow us for the latest news.
  
  
• Events of interest: Check out upcoming conferences, trade shows, and webcasts that are of interest to IBM open source developers.
  
  
• developerWorks Open source zone: Find extensive how-to information, tools, and project updates to help you develop with open source technologies and use them with IBM's products, as well as our most popular articles and tutorials.
  
  
• developerWorks On demand demos: Watch our no-cost demos and learn about IBM and open source technologies and product functions.
  
  

Get products and technologies

• IBM trial software: Innovate your next open source development project using trial software, available for download or on DVD.
  
  

Discuss

• developerWorks community: Connect with other developerWorks users while exploring the developer-driven blogs, forums, groups, and wikis.
  
  

About the author

Avinesh Kumar is a technical lead of the Andrew File System Team at the IBM Software Labs in Pune, India. He has worked for IBM for the past five years. His current projects include enhancing IBM AFS® with new features and porting IBM AFS to newer releases of RedHat Enterprise Linux. He works with kernel- and user-level debugging dumps and crashes, as well as reported bugs on the Linux, AIX, and Solaris platforms. Avinesh has an MCA from the Department of Computer Science at the University of Pune. He is a Linux enthusiast who spends his spare time exploring the Linux kernel on his Maverick Ubuntu box.

Rate this article

Comments

Back to top

Table of contents

• Overview
• Brief introduction to AFS
• Basic AFS concepts
• Brief introduction to Kerberos
• Kerberos 5 configuration
• Getting OpenAFS source and doing a build of OpenAFS
• Configuration of AFS cell
• AFS server configuration
• AFS client configuration
• Final step
• All done!
• Summary
• Acknowledgment
• Resources
• About the author
• Comments

Local resources

• IBM Innovation Center - Austin, TX
• IBM Innovation Center - Chicago, IL
• IBM Innovation Center - Dallas, TX
• IBM Innovation Center - San Mateo, CA
• IBM Innovation Center - Waltham, MA
• Technical Briefings in North America

Dig deeper into Open source on developerWorks

Try IBM PureSystems. No charge.

Experience today!

---

Get started with the cloud-based trial and pattern development kit.

Special offers