Using I/O Completion Ports with AIO Requests

To use I/O completion ports (IOCP) with AIO requests, set the AIO_EXTENDED flag in the aio_flags field and the aio_version field to a value of AIOCBX_VERS2 or higher. All other extended fields defined must be set to 0 if they are not used.

The following fields are used with this extended functionality:

Field Version
aio_iocpfd AIOCBX_VERS2

A limitation of the AIO interface that is used in a threaded environment is that aio_nwait() collects completed I/O requests for ALL threads in the same process. In other words, one thread collects completed I/O requests that are submitted by another thread. Another problem is that multiple threads cannot invoke the collection routines (such as aio_nwait()) at the same time. If one thread issues aio_nwait() while another thread is calling it, the second aio_nwait() returns EBUSY. This limitation can affect I/O performance when many I/Os must run at the same time and a single thread cannot run fast enough to collect all the completed I/Os.

Using I/O completion ports with AIO requests provides the capability for an application to capture results of various AIO operations on a per-thread basis in a multithreaded environment. This functionality provides threads with a method of receiving completion status for only the AIO requests initiated by the thread.

The IOCP subsystem only provides completion status by generating completion packets for AIO requests. The I/O cannot be submitted for regular files through IOCP.

The current behavior of AIO remains unchanged. An application is free to use any existing AIO interfaces in combination with I/O completion ports. The application is responsible for "harvesting" completion packets for any noncanceled AIO requests that it has associated with a completion port.

The application must associate a file with a completion port using the CreateIoCompletionPort IOCP routine. The file can be associated with multiple completion ports, and a completion port can have multiple files associated with it. When making the association, the application must use an application-defined CompletionKey to differentiate between AIO completion packets and socket completion packets. The application can use different CompletionKeys to differentiate among individual files (or in any other manner) as necessary.

The application must also associate AIO requests with the same completion port as the corresponding file. It does this by initializing the aio_iocpfd of the AIOCB with the file descriptor of the completion port. An AIOCB can be associated with only one completion port, but a completion port can have multiple AIOCBs associated with it. The association between a completion port and an AIOCB must be done before the request is made. This is accomplished using an AIO routine, such as aio_write, aio_read, or lio_listio. If the value in the aio_iocpfd field is not a valid completion port file descriptor, the attempt to start the request fails and no I/O is performed.

An association must be made directly between a completion port and an AIOCB. For example, if you want to call lio_listio(), each AIOCB in the lio_listio chain must be associated individually prior to the call. It is not necessary to have all AIOCBs in the chain associated with a completion port.

After an association is made, it remains until the application explicitly clears it by using a value of 0 for the aio_iocpfd field, or the AIOCB is destroyed. A completion packet is created only when I/O completes for an AIOCB that has been associated with a completion port.

A summary of the steps that an application takes to use I/O completion ports with AIO requests is as follows:
  1. Opens a regular file for I/O.
  2. Calls the CreateIoCompletionPort routine to create an I/O completion port (IOCP), using the file descriptor for the regular file and an application-defined CompletionKey, which is used to differentiate AIO requests from socket I/O. The CreateIoCompletionPort function returns an IOCP file descriptor that corresponds to the newly created IOCP.
  3. Allocates and clears (using the bzero function) an AIO control block. Indicates that I/O completion ports are to be used with AIO requests by setting the AIO_EXTENDED flag of the AIOCB's aio_flags field. Also sets the aio_version field to a value of AIOCBX_VERS2 or higher.
  4. Associates the AIO request with the IOCP by initializing the aio_iocpfd field in the AIOCB to contain the IOCP file descriptor returned by the CreateIoCompletionPort routine.
  5. Starts the AIO request using existing AIO interfaces. Multiple requests can be started using the lio_listio interface.
  6. Calls the GetQueuedCompletionStatus function with the IOCP file descriptor to collect the results of the completed AIO requests on a particular IOCP. The application provides the address of a pointer in the LPOVERLAPPED argument to GetQueuedCompletionStatus, so the corresponding AIOCB pointer can be returned. Details of the AIO request can be determined by examining the returned AIOCB.
  7. After all I/O is complete, the application is responsible for closing all file descriptors.