Raw I/O

About this task

The index has been designed to support raw I/O devices. These are devices for which the application can take complete control. Raw I/O is much more efficient than writing to files in a filesystem. In this section, specific examples are based on recent Linux systems. Most Linux systems offer similar capabilities. To use raw I/O you will need to perform the following steps:

Procedure

  1. Determine the amount of space that your index will require. An easy way to do this is to build an index. You probably want to add a factor to allow for growth if you will be updating the index.
  2. Create two disk partitions of this size. For example, partitions /dev/sda4 and /dev/sda5.
  3. (On Linux only) Create raw devices. For example: raw /dev/raw/raw1 /dev/sda4; raw /dev/raw/raw2 /dev/sda5
  4. Remove the existing data for that search collection from the search collection's Overview page.
  5. The Overview page contains meta information which allows you to specify the index file and the staging index file. Specify one of the two raw devices for each of these two values.

Results

If you are using software RAID in Linux, it seems that the default configuration uses a 1024-sector read-ahead, which is very detrimental to performance. We have included a utility to change this. The following command sets the read-ahead to a more reasonable value:

 /usr/local/vivisimo/bin/readahead /dev/md0 0