Crawling and Indexing

Once a collection is created, it is ready to receive data for indexing from the enqueueing functions. These functions will automatically start the crawler service if necessary. The crawler service is the service that retrieves data from specific URLs, and processes that data so that it can be indexed.

Note: The search-collection-enqueue, search-collection-enqueue-deletes, search-collection-enqueue-xml, and collection-broker-enqueue-xml enqueueing functions provide a Boolean exception-on-failure parameter that enables you to specify whether the failure of any enqueue operation will cause a search-collection-enqueue exception to be thrown. The default value for this parameter is false, which means that exceptions will not be raised for errors associated with individual crawl-url elements if those errors are not the result of the crawler's internal state or problems communicating with the crawler. Setting this parameter to true will cause an exception to be raised for any type of enqueue failure.

If the error attribute is set on any crawler-service-enqueue-response node that is returned by any of these functions, an exception will be raised regardless of the value of the exception-on-failure parameter. The error attribute is set for enqueue operations that fail due to problems communicating with the crawler, the internal state of the crawler, or because of invalid XML content. See Handling Enqueueing Errors for more information.