eXtreme Scale enhances Apache Lucene text search engine
TedKirby 0600004ET8 Visits (3106)
"Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires full-text search, especially cross-platform. Apache Lucene is an open source project available for free download." Many applications and web sites use and/or are powered by Lucene: list.
Lucene maintains an index for rapidly searching documents. The index may be on disk or in memory. An index may be copied from one Directory to another, say from disk to memory.
Billy Newport has written code to allow a Lucene index (Directory) to be stored in an eXtreme Scale grid. Billy announces and describes his code in his blog, starting with this entry. Source code and binaries are provided, along with usage instructions. Lucene indexes may be created in a grid, or existing indices on disk may be loaded to the grid for faster searching.
Searching an index in memory is faster than searching one on disk. Putting the index in an eXtreme Scale grid provides a number of advantages over traditional memory: