IBM AIX device driver development

A tutorial on AIX device driver framework and related APIs

This tutorial illustrates the IBM® AIX® device driver framework and related application programming interfaces (APIs). Here, we shall discuss about the device switch structure, API to register device switch methods, major and minor number management, and moving data from kernel to user space.

Share:

Gautam Raut (gautraut@in.ibm.com), Senior Software Engineer, IBM China

GautamGautam Raut works as a Senior Software Engineer for the Andrew File System (AFS) Team at the IBM Software Labs in Pune, India. He works with kernel and user-level debugging of dumps and crashes, as well as reported bugs on the Linux and AIX platforms. He has also worked on various AFS-specific AIX kernel components. Gautam holds M.S. in Computer Science from the University of Pune. He is a Linux enthusiast who spends his spare time exploring the Linux kernel on his Fedora 8 box.



28 March 2013

Also available in Chinese

Introduction

In traditional UNIX®, the term device refers to hardware components, such as disk drives, tape drives, printers, keyboards, pseudo devices (such as the console, error special file and null special file), and so on. In AIX, these devices are referred to as kernel devices, which have device drivers and are known to the system by major and minor numbers.

AIX device drivers are a type of AIX kernel extensions. Kernel extensions run within a protected domain of a kernel. They can be loaded into kernel during system start or run time and removed at run time. User-level code can access registered device driver code only through system calls. Device drivers add extensibility, configurability and ease of system administration to AIX.

To understand basic kernel extension development, refer to the IBM developerWorks article, Writing AIX kernel extensions.

Device drivers use core kernel services exported by AIX kernel "/unix" by referring to /usr/lib/kernex.exp. "kernex.exp" contains a list of symbols exported by kernel. These exported symbols are essentially kernel functions and storage locations (kernel global data structure). The linker program (ld) uses this information while linking the compiled device driver code.


Types of devices

  • Character
  • Block
  • Streams
  • Network
  • Multiplexed character device

Devices are represented as special type of files listed under the /dev directory.

These files do not have any printable content in them. Rather, they provide an interface for users to interact with the associated device. Inodes of these files include major and minor numbers. These files are created using the mknod() system call.


Object Data Manager (ODM)

ODM maintains data represented in form of classes of objects. It is used for storing device-specific configuration information. It has predefined database for storing configuration data for all devices. It also has customized database for device instances that are currently running in the system. ODM needs to be initialized before use. Application intending to configure device driver, must initialize ODM first.

Application initializes ODM using the odm_initialize() API defined in /usr/include/odmi.h.


Basics of kernel extension configuration

Components and use of struct cfg_load

Structure for loading a kernel extension:

struct cfg_load
{
   caddr_t path; /* ptr to object module pathname */
   caddr_t libpath; /* ptr to a substitute libpath */
   mid_t   kmid;    /* kernel module id (returned) */
};

Components and use of struct cfg_dd

Structure for calling a device driver's config (module) entry point:

struct cfg_dd
{
  mid_t   kmid;/* Module ID of device driver */
  dev_t   devno; /* Device major/minor number */
  int     cmd; /* Config command code for device driver*/
  caddr_t ddsptr; /* Pointer to device dependent structure*/
  int     ddslen; /* Length of device dependent structure */
};

Sysconfig() system call

Sysconfig() requires three parameters.

int sysconfig(             
   int cmd, /* Command to be executed */
   void *parmp,/* Address of structure containing info for cmd */
   int parmlen /* Length of parmp information */
 );

Sysconfig() is used to control the lifecycle of a kernel extension. As mentioned in /usr/include/sys/sysconfig.h, the following commands are passed to sysconfig() as cmd parameter.

  • SYS_KLOAD: Loads a kernel extension object file in a kernel memory.
  • SYS_SINGLELOAD: Loads a kernel extension object file only if it is not already loaded.
  • SYS_QUERYLOAD: Determines if a specified kernel object file is loaded.
  • SYS_KULOAD: Unloads a previously loaded kernel object file.
  • SYS_CFGKMOD: Calls the specified module at its module entry point for configuration purposes.

Device driver basics

Major and minor numbers

Devices are treated as special files under the /dev directory. Hence, every device has a name and an associated inode or index node under its root file system. The file system entry point of every device contains major and minor numbers.

A major number is used to find this device uniquely in the system. It is an index into device switch table. Device switch table contains pointers to the device-specific methods that are essentially implementations of generic file system functions. In this manner, a file system call issued by a user process on a given device is resolved into a call to the appropriate device driver function. A minor number is used by the driver internally to distinguish between logical channels for multiplexed devices.

A major number can be obtained by a library call, genmajor().

Syntax of genmajor() is:

	int genmajor(char* name_of_device_driver)

A minor number can be obtained by a library call, genminor().

It generates either the smallest unused minor number available for a device, a preferred minor number if it is available, or a set of unused minor numbers for a device.

Syntax of genminor() is:

int *genminor (
 char * device_instance,
 /*
 Points to a character string containing the device instance name.*/
 int major_no,
 /*
  The major number of the device instance.*/
 int preferred_minor,
 /*
  Contains a single preferred minor number or a starting  
minor number for generating a set of numbers. */
 int minors_in_grp,
 /*
 Indicates how many minor numbers are to be allocated.	*/
 int inc_within_grp,
 /*
 Indicates the interval between minor numbers.*/
 int inc_btwn_grp
 /*
 Indicates the interval between groups of minor numbers.*/
);

makedev() API

Major and minor numbers for a given device needs to be packaged into a 32- or 64-bit integer, depending on the architecture. This integer has a data type dev_t. makedev() is a macro that creates dev_t from the given major and minor numbers. It is defined under <sys/sysmacros.h>. makedev64() is used to create dev_t for a 64-bit environment.

Syntax of makedev() or makedev64() is:

 dev_t makedev64(int major, int minor);

mknod() system call

It is defined in <sys/stat.h>. It is used to create an ordinary file, first-in-first-out (FIFO), or a special file. It requires root privilege to use mknod() to create a device special file.

Syntax of mknod() is:

 int mknod (
  char *Path,
  /*
   Names the new device special file.
  */
  int Mode,
  /*
   Specifies the file type, attributes, and access
   permissions.
  */
  dev_t Device
  /*
   Device number generated by makedev() subroutine.
  */
 );

Device switch table

Device switch table is an array of struct devsw structures. It is indexed by major number and pinned in memory (that is, it will never be swapped out of RAM). Block and character devices use this kernel structure for registering themselves with the root file system.

Device switch table entry structure:

struct devsw
{
 int (*d_open)(); /* entry point for open routine */
 int (*d_close)(); /* entry point for close routine */
 int (*d_read)();/* entry point for read routine */
 int (*d_write)(); /* entry point for write routine */
 int (*d_ioctl)();/* entry point for ioctl routine */
 int (*d_strategy)();/* entry point for strategy routine */
 struct tty  *d_ttys;/* pointer to tty device structure */
 int (*d_select)();  /* entry point for select routine */
 int (*d_config)();  /* entry point for config routine */
 int (*d_print)(); /* entry point for print routine */
 int (*d_dump)(); /* entry point for dump routine */
 int (*d_mpx)(); /* entry point for mpx routine */
 int (*d_revoke)(); /* entry point for revoke routine */
 caddr_t d_dsdptr; /* pointer to device specific data */
 /*
  * The following entries are control fields managed
  * by the kernel and should not be modified by device
  * drivers or kernel extensions.  They may be set to
  * 0 in the devsw structure for devswadd(), since they
  * are not copied into the device switch table.
  */
 caddr_t d_selptr;/* ptr to outstanding select cntl blks*/
 ulong   d_opts;/* internal device switch control field */
};

Driver entry points

Driver entry points are nothing but the members of the struct devsw structure. All these members must be initialized before adding the device into device switch table. It is not mandatory for a device driver to implement all the methods. An unimplemented member can be initialized to nodev. These entry points can accept dev_no (device number of the device or the subdevice to which this operation is directed), chan (channel ID for a multiplexed device), ext ( an integer useful for calls to extended subroutines such as openx, readx, writex and ioctlx that pass extra device-specific parameters to few of the device entry points.)

Listing 1. 1) ddconfig or d_config:
 int d_config(dev_t dev_no, int cmd, struct uio *uiop)

It is invoked by the sysconfig() system call. It prepares a device for its first open() call. It can initialize, terminate, request configuration data for device or perform device specific configuration function. The uio structure contains data area for configuration information.

Listing 2. 2) ddopen or d_open:
int d_open(dev_t dev_no, ulong flag, chan_t chan, ext_t ext)

It enables device operation and prepares for data transfer. It allocates internal buffers and enforces policies with respect to how a device can be opened based on the current device state. This is invoked by the open() or create() system calls and fp_open() or fp_opendev() kernel service. The input parameter flag specifies open file control flags, such as DREAD, DWRITE, and so on.

Listing 3. 3) ddclose or d_close:
int d_close(dev_t dev_no, chan_t chan)

It closes a previously opened device instance. It is called by the close() system call or the fp_close() kernel service. A device instance is considered closed after d_close() returns to the caller even if a non-zero return code is returned.

Listing 4. 4) ddread or d_read:
 int d_read(dev_t devno, struct uio *uiop, chan_t chan, int ext)

It reads in data from a character device. It is called by system calls, such as read() or readx() and the fp_rwuio() kernel service. Here, the uio structure describes the data area or areas in which to be written.

Listing 5. 5) ddwrite or d_write:
int d_write (dev_t devno, struct uio *uiop, chan_t chan, int ext)

It writes out data to a character device. It is called by system calls, such as write() or writex(), and the fp_rwuio() kernel service. Here, the uio structure describes the data area or areas from which to be written.

Listing 6. 6) ddioctl or d_ioctl:
int d_ioctl(dev_t devno, int cmd, void *arg, ulong devflag, chan_t chan, int ext)

It performs special I/O control operations requested in ioctl() or ioctlx() system calls or the fp_ioctl() kernel service. It must respond to the IOCINFO command which returns the devinfo structure that describes the device.

Listing 7. 7) ddstrategy or d_strategy:
int d_strategy(struct buf* buffer)

It performs block-oriented I/O by scheduling a read or write to a block device. It maps I/O requests to device requests so that with minimum device requests maximum data transfer is achieved. Buffer is a pointer to a linked list of buffer structures chained with the b_forw pointer. The ddstrategy routine can receive a single request with multiple buf structures. However, it is not required to process requests in any specific order. This routine never returns a return code and never waits for I/O completion.

Listing 8. 8) ddselect or d_select:
int d_select(dev_t devno, ushort events, ushort *reventp,  int chan)

It checks for one or more events specified by the events flag occurred on a given device and returns a pointer to the occurred events in reventp. It is called by select and poll system calls or fp_select kernel service.

Listing 9. 9) d_mpx or ddmpx:
int d_mpx(dev_t devno, chan_t* chanp, char* channame)

It allocates and deallocates logical channels for multiplexed devices. It is called once for each open() of a device file before the d_open call to allocate a channel and once for each close of a device file after d_close. It is supported by only character class device drivers. chanp is a pointer to channel ID and channame is a path name extension for the channel to be allocated.

Listing 10. 10) d_revoke or ddrevoke:
int d_revoke (dev_t devno, chan_t chan, int flag)

For driver for devices requiring trusted computing path, ddrevoke() provides a secure path to the terminal. It is supported only by character class device drivers. It is called by the revoke() system call or the frevoke() kernel API.

Listing 11. 11) d_dump or dddump:
int d_dump(dev_t devno, struct uio * uiop, int cmd, int arg, chan_t chan, int ext)

It writes system dump data to a device. This is an optional routine for a device driver. It is required only when the device driver supports a device as a target for a possible kernel dump. This routine must not call any kernel service that can page fault.


Device configuration routine

Listing 12. 1) devswadd kernel service:
int devswadd (dev_t devno, struct devsw *dswptr)

This adds a device entry pointed by dswptr for a device specified by devno to device switch table. It is typically called by the ddconfig() routine of device driver.

Listing 13. 2) devswdel() kernel service:
int devswdel(dev_t devno)

This deletes a device driver entry from the device switch table. It is called by the ddconfig() routine for termination of device driver.


Components and significance of the uio structure

The uio structure contains a memory buffer used for exchange of data between user and kernel space while implementing driver routines. The uio structure describes a buffer that is not contiguous in virtual memory. The ureadc, uwritec, uiomove, and uphysio kernel services all perform data transfers into or out of a data buffer described by the uio structure. The uio structure is defined in /usr/include/sys/uio.h file.

struct uio {
/* ptr to array of iovec structs describing  user buffer for data transfer */
   struct  iovec *uio_iov;
/* ptr to array of xmem structs containing cross memory descriptors for iovec array.*/	
   struct  xmem  *uio_xmem;
/* #iovec elements remaining to be processed*/
   int32long64_t  uio_iovcnt;

/* #iovec elements already processed */
   int32long64_t  uio_iovdcnt;
#ifdef _LONG_LONG
/* byte offset in file/dev to read/write */
   offset_t uio_offset;    
#else /* _LONG_LONG */
#ifdef __64BIT__
/* off_t offset for ANSI-C mode */
   off_t   uio_offset;     
#else
/* ANSI-C does not support long long */
   int     uio_rsvd;
/* off_t offset for ANSI-C mode      */       
   off_t   uio_offset;     
#endif /* __64BIT__ */
#endif  /* _LONG_LONG */
/* Byte count for data transfer  */
   int32long64_t uio_resid;                
/* Type of buffer being described by uio structure. Data pointed by 
  buffer can either be in user or kernel or cross-memory region. */
   short   uio_segflg;
/* copy of file modes from open file structure */    
   long    uio_fmode;      
};

Example of a device driver

sample_driver.c

#include <stdio.h>
#include <syslog.h>
#include <sys/types.h> /* for dev_t and other types */
#include <sys/errno.h> /* for errno declarations */
#include <sys/sysconfig.h> /* for sysconfig() */
#include <sys/device.h> /* for devsw */
#include <sys/uio.h> /* for uiomove */
#include <sys/sysmacros.h>

struct dr_data
{
  char buffer[1024];
}dr_data[5];

int dr_open(dev_t devno, ulong devflag, chan_t chan, int ext)
{
   bsdlog(LOG_KERN|LOG_DEBUG,"Inside dr_open\n");
   return 0;
}


int dr_close (dev_t devno, chan_t chan)
{
   bsdlog(LOG_KERN|LOG_DEBUG,"Inside dr_close \n");
   return 0;
}

int dr_read (dev_t devno, struct uio *uiop, chan_t chan, int ext)
{
   uint min;
   int rc;

   min = minor_num(devno);
   rc = uiomove(dr_data[min].buffer, 1024, UIO_READ, uiop);
   bsdlog(LOG_KERN | LOG_DEBUG, "Inside dr_read min: 
   %d, buffer: %s \n", min, dr_data[min].buffer);
   return rc;
}

int dr_write (dev_t devno, struct uio *uiop, chan_t chan, int ext)
{
   uint min;
   int rc;

   min = minor_num(devno);
   rc = uiomove(dr_data[min].buffer, 1024, UIO_WRITE, uiop);
   bsdlog(LOG_KERN | LOG_DEBUG,"Inside dr_write min: 
   %d, buffer: %s \n", min, dr_data[min].buffer);
   return rc;
}

int driverdd_config (dev_t devno, int cmd, struct uio *uiop)
{
   struct devsw dswp;
   int rc = 0;

   switch (cmd)
   {
       case CFG_INIT:
            dswp.d_open     = dr_open;
            dswp.d_close    = dr_close;
            dswp.d_read     = dr_read;
            dswp.d_write    = dr_write;
            dswp.d_ioctl    = nodev;
            dswp.d_strategy = nodev;
            dswp.d_ttys     = NULL;
            dswp.d_select   = nodev;
            dswp.d_config   = driverdd_config;
            dswp.d_print    = nodev;
            dswp.d_dump     = nodev;
            dswp.d_mpx      = nodev;
            dswp.d_revoke   = nodev;
            dswp.d_dsdptr   = NULL;
            dswp.d_selptr   = NULL;
            dswp.d_opts     = DEV_MPSAFE|DEV_64BIT;



            if((rc = devswadd(devno, &dswp)) != 0)
            {
                rc = major_num(devno);
                printf("Error in devswadd: %d\n", rc);
                return rc;
            }
            break;

        case CFG_TERM:
            if((rc = devswdel(devno)) != 0)
            {
                printf("Error in devswdel: %d\n", rc);
                return rc;
            }
            break;
        default:
             printf("Invalid command \n");
            return EINVAL;
    }
    return 0;
}

Example of a configuration application for a given device driver

Config_mgr.c

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/errno.h>
#include <sys/sysmacros.h>
#include <sys/sysconfig.h>
#include <sys/device.h>
#include <sys/mode.h>
#include <odmi.h>
#include <cf.h>
#include <sys/cfgodm.h>
#include <sys/cfgdb.h>

int main()
{
   struct cfg_load ext_load; /* to load kernel extension */
   struct cfg_dd ddcfg; /* to invoke driver config() */
   char c, str[80];
   int rc;
   int major, minor = 0;
   dev_t devno;

   rc = odm_initialize();

   printf("\n Enter choice, (l)oad, (u)nload \n");
   while((c = getchar()) < 'a' && c > 'z');

   switch(c) {
      case 'l':
          ext_load.path = "sample_driver";
          ext_load.libpath = NULL;
          ext_load.kmid = 0;

          if(sysconfig(SYS_KLOAD,
          &ext_load, sizeof(struct cfg_load))) {
             printf("Error in loading extension\n");
             exit (1);
          }
          else
             printf("Extension Successfully loaded, kmid is %d\n", ext_load.kmid);

          major = genmajor("sample_driver");
          printf("Major number: %d\n", major);
          devno = makedev64(major, minor);
          ddcfg.kmid = ext_load.kmid;
          ddcfg.devno = devno;
          ddcfg.cmd = CFG_INIT;
          ddcfg.ddsptr = NULL;
          ddcfg.ddslen = 0;

          if (rc = sysconfig(SYS_CFGDD,
             &ddcfg, sizeof(ddcfg))) {
             printf("Error in configuring device %d %d\n", rc, errno);
             exit (1);
          }

          for(minor = 0; minor <=2; minor++) {
             devno = makedev64(major, minor);
             sprintf(str, "/dev/drvdd%d", minor);
             if (mknod(str, 0666 | _S_IFCHR, devno) == -1){
               printf("Error while creating device %s\n", str);
               exit (1);
             }
          }
          break;

    case 'u':
         ext_load.path = " sample_driver";
         ext_load.libpath = NULL;
         ext_load.kmid = 0;

         if(sysconfig(SYS_QUERYLOAD, &ext_load,
         sizeof(struct cfg_load)))
            printf("Error while querying\n");

         if(sysconfig(SYS_KULOAD, &ext_load,
         sizeof(struct cfg_load)))
            printf("Error in unloading extension\n");
         else
            printf("Extension Successfully unloaded\n");
         break;


    default:
         printf("Incorrect option\n");
         break;
 }
        return 0;
}

Example of an application making use of an implemented device driver

application.c

#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>

int main()
{
  int fd0, fd1;
  char rbuf[1024];

  fd0 = open("/dev/drvdd0", O_RDWR);
  if(fd0 < 0){
    printf("Error in opening /dev/drvdd0\n");
    return -1;
  }
  fd1 = open("/dev/drvdd1", O_RDWR);
  if(fd1 < 0){
    printf("Error in opening /dev/drvdd1\n");
    return -1;
  }
  write(fd0, "Hello", 5);
  write(fd1, "World", 5);

  read(fd0, rbuf, 5);
  printf("Read from /dev/drvdd0 : %s\n",rbuf);

  read(fd1, rbuf, 5);
  printf("Read from /dev/drvdd1 : %s\n",rbuf);
  return 0;
}

Makefile

all: sample_driver config_mgr application

config_mgr: config_mgr.c
          cc -q64 -o config_mgr -g config_mgr.c -lodm -lcfg

application: application.c
	  cc -o application application.c

K_LIBS= -bI:/usr/lib/kernex.exp -lsys -lcsys

sample_driver: sample_driver.c
	  cc -q64 -o sample_driver64.o -c sample_driver.c -D_KERNEL -D_64BIT_KERNEL
	  ld -b64 -o sample_driver sample_driver64.o -e driverdd_config $(K_LIBS)
        
clean:
	  rm -f *.o sample_driver sample_driver32 
        sample_driver64 config_mgr application  2> /dev/null

Compilation and testing of a sample driver

Log in with root privilege on to your system with AIX 6.1 or a later version. Copy above files in your development directory and run the make command from the command prompt of shell.

This builds the necessary driver, configuration utility, and an application that in turn invokes the driver APIs. Then run the ./config_mgr utility from the command prompt and check whether the driver is getting loaded successfully. If no errors are thrown, you can proceed with ./application, which will test the driver APIs.

Resources

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into AIX and Unix on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=AIX and UNIX, Linux
ArticleID=861832
ArticleTitle= IBM AIX device driver development
publish-date=03282013