Tokens available locally but data must be read

The second, more complex, type of read operation is necessary when the data is not in memory.

This occurs under three conditions:

The token is acquired on a previous read that found no contention.
The buffer is stolen for other uses.
On some random read operations.
In the first of a series of reads, the token is not available locally, but in the second read it might be available.

In such situations, the buffer is not found and must be read from disk. No token activity has occurred because the node has a sufficiently strong token to lock the required region of the file locally. A message is sent to the daemon, which is handled on one of the waiting daemon threads. The daemon allocates a buffer, locks the file range that is required so the token cannot be stolen for the duration of the I/O, and initiates the I/O to the device holding the data. The originating thread waits for this to complete and is posted by the daemon upon completion.