Document Level Security

Document level security means that the current user's authentication information will be used to filter the results that they see on a document-by-document basis. That is, only documents that they are authorized to view will be viewable. Every search engine source defines user variables for their authorization username and password. The default value of these variables will be the user's login name and password. These passwords are only ever communicated to the query-service using an encrypted representation.

To enable document level security checks for a result, the crawler option authorization-required must be specified for the URL which generates the document element. If specified for this URL then an optional authorization-url may also be specified. If it is not specified then the authorization-url will be the URL that was actually crawled.

When a document with an authorization URL is found as the result of a search, the query-service will issue a HEAD request of the authorization URL using the user's credentials. If there is an error retrieving the page (including a 404, not found error), the result will be dropped. If it is fetched successfully, the user will be allowed to see the result.

A separate authorization URL is important if you want to bulk authorize many pages. For example, if you have the same authorization for all pages in http://hr.vivisimo.com, you could simply use the ability to retrieve that URL as the authorization for the entire site. This is important because document level security is an expensive proposition if 200 search results must be validated.

If you want to allow users to specify their own username and password for a given collection (rather than using their login information), you need to edit the source, edit the declarations and set the user variable flag for authorization-user and authorization-password. This will allow users to modify the credentials in the user profile.

Warning : Currently there is no way to verify that a file URL is retrievable. To crawl a Windows (SMB) share, you should enable "Web Sharing" on the share, and not mount it if you want to be able to check authorization of the individual files.