IBM OmniFind Enterprise Edition is a full-text enterprise search product designed to provide superior performance, scale, and result quality with a broad range of data source support. Of the Lotus support, OmniFind can securely index and search Lotus Notes databases, Lotus QuickPlace®, and Lotus Domino® Document Manager.
OmniFind enhances an application's search functionality beyond what is provided by Domino today in several important ways. First, OmniFind can extend your reach outside of Domino to include content from the Web, file systems, relational databases, and other mail and document management systems. The end user's search experience can be enriched with OmniFind's advanced search features such as spell checking, dynamic summaries, quick links, site collapse, and much more. On the backend, OmniFind can improve the performance of your Domino servers by off-loading full text indexing and search operations to OmniFind. And OmniFind provides true enterprise scale, supporting up to 20 million documents.
You may have already acquired OmniFind and discovered how quick and easy it is to set up an index and start searching. You may have also discovered that the client search applications provided by OmniFind are J2EE based, intended to be run within a J2EE application server environment such as IBM's WebSphere®. This may be suitable for your client base if access is via a Web browser, but perhaps your end users rely exclusively on the Notes Client to access Domino. In that case, read on to find out how to add search functionality to your conventional Lotus Notes applications.
OmniFind sample search application design
An important aspect of the design is the relationship of OmniFind to Domino. OmniFind runs in its own process space separate from Domino. Consequently, it is desirable and most often recommended that OmniFind run on its own dedicated server. This is to support the potentially high number of enterprise documents to be indexed and also has the benefit of relieving Domino of all full text indexing and search overhead. The topography of servers and the Notes client are illustrated in Figure 1.
Figure 1. OmniFind Notes application topography
OmniFind is installed on a separate server and is configured to crawl and index one or more Notes databases in a given Domino domain. These databases are made available for search through OmniFind's search index. This is shown in the lower left hand corner of Figure 1. It is important to note that the user is not searching the original Notes content but rather the OmniFind search index which can contain other crawled content from the enterprise (for example, Web, file system, and so on).
Our OmniFind search application is designed as a conventional Notes database named OmniFind.nsf and is shown in the upper left hand corner of Figure 1. OmniFind.nsf consists of two forms:
- One form allows you to indicate which OmniFind system to use for searching (more on this later).
- The second is the actual search form used for submitting queries and displaying search results. Accompanying the search form is LotusScript that is responsible for calling SIAPI. SIAPI communicates with OmniFind using the settings provided in the first configuration form
Figure 2 presents a screen snapshot of the OmniFind search application. The table of contents pane on the left provides three options. The OmniFind Settings option takes you to the configuration form shown in Figure 3. Here you provide the host and port of the OmniFind server and a user Id and password if security is enabled on the OmniFind server. An Application Id is needed as well as a Collection Id. To keep things simple, this particular design searches only one OmniFind collection denoted by the assigned Collection Id. A single OmniFind collection can contain documents from multiple Notes databases and other sources. A natural extension of this design would be to allow the user to select from a list of collections they would like to search. A Test Connection button is provided that attempts to communicate with the OmniFind server using the settings provided. A status message is displayed whether the connection was successful or not.
Figure 2. OmniFind Notes application
The Saved Searches option presents a list of previously executed queries and their results. This is a commonly requested feature of a typical search application and is easily provided by the inherent design of Notes. Because the search form is just that, a Lotus Notes form, it is possible to save an instance of it as a document within the OmniFind.nsf database. A database view of these search forms is provided that lists the author of the query, the date and time the query was submitted and the keywords used in the query. If the OmniFind.nsf database were placed on a Domino server then it is possible to have these saved queries shared with other users potentially eliminating the need to issue a search altogether.
The Start Searching option in the table of contents takes you to the search form. Figure 2 shows an instance of the search form in the lower right hand corner. The form consists of two parts. The top part provides an edit box where the user enters their search terms followed by a search button to execute the search. The results are displayed in the bottom half of the form below the horizontal rule. The results are displayed ten per page so there are previous and next buttons for navigating through the various pages of results.
Figure 3. OmniFind settings form
Installing the OmniFind sample Notes application
It is assumed that the OmniFind product has already been installed on a separate server and configured with a searchable collection (index). The collection can contain documents crawled from any supported source. However the OmniFind sample application is currently designed to only display icons representing results from Lotus Notes and file system documents. This is only a restriction on showing what types of results are being displayed. You can add more source type icons at your leisure.
Installation of the OmniFind sample search application is a two step process.
- Download the OmniFind.nsf database from the Downloads section of this article and copy it into the data directory of your Domino data directory.
- Copy the esapi.jar and siapi.jar files from the OmniFind <installdir>/lib directory into the <DominoDir>/jvm/ext directory
The OmniFind sample application uses LotusScript to call the IBM Search and Index API (SIAPI). The esapi.jar and siapi.jar files are necessary to support this and must be accessible by Domino in the jvm/ext directory.
Verification of the installation is a simple matter of accessing the OmniFind.nsf database using the Notes client, filling out the OmniFind settings form and then issuing searches using the search form. It is always a good idea to press the Test Connection button on the settings form to verify that the settings are correct. Common mistakes are to not provide a correct collection Id, host or port. The ApplicationId by default should be set to Default.
A simple SIAPI example In Java
Before taking a look at how to call SIAPI from within LotusScript it is best if we review a simple search example using SIAPI in Java alone. This way you can see what is necessary by way of SIAPI calls to issue a search. Once you are familiar with these requirements the rest is a matter of making these same calls in LotusScript using the LotusScript-to-Java interface. Listing 1 illustrates an abbreviated coding example of a SIAPI search.
Listing 1. SIAPI example in Java
// create a valid Application ID that will be used
// by the Search Node to authorize access to the collection
ApplicationInfo appinfo = factory.createApplicationInfo(applicationName);
appinfo.setPassword(applicationPassword);
// create a new Properties object.
Properties config = new Properties();
config.setProperty("hostname", "OmniFindHostName");
config.setProperty("port", "80");
config.setProperty("timeout", "60");
config.setProperty("username", "websphereUser");
config.setProperty("password", "webspherePassword");
// obtain the OmniFind specific SIAPI Search factory implementation
SearchFactory factory = (SearchFactory)
Class.forName("com.ibm.es.api.search.RemoteSearchFactory").newInstance();
// obtain the Search Service implementation
SearchService searchService = factory.getSearchService(config);
// obtain a Searchable object to the specified collection ID
Searchable searchable = null;
try { searchable = searchService.getSearchable(appinfo, collectionId);
} catch (SiapiException e) {return;}
// create a new Query object using the specified query string
Query q = factory.createQuery("search terms go here");
// execute the search. A ResultSet object will be returned
ResultSet rset = null;
try { rset = searchable.search(q);
} catch (SiapiException e) { return; }
Result r[] = rset.getResults();
for (int k = 0; k < r.length; k++) {
System.out.println("Result " + k + ": " + r[k].getDocumentID());
}
|
An ApplicationInfo object is first created and is used to identify your program to OmniFind. An OmniFind administrator can assign all or a subset of collections that are authorized to be searched by your program.
Next a Properties object is created with connection information to the OmniFind server. This Properties object is used on the getSearchService call to obtain an actual connection with the OmniFind service. Note that an OmniFind search factory must first be instantiated to get the OmniFind implementation.
Once you have a SearchService object, you can obtain one or more searchable objects each associated with an authorized collection that can be searched. In this case we are only obtaining a single searchable object.
Next we create a query object for the given search phrase. It is on the createQuery call that you would provide your own query expression.
We are now ready to issue the search using the search method of the searchable object and passing it the query object previously created. The invocation executes the search and returns a SIAPI result set object. The individual results are returned as an array of results using the getResults call. There are many access methods for the different attributes of a single result such as obtaining the document Id, Title, and/or URL.
The remainder of this article will describe in detail the LotusScript used in OmniFind.nsf. After learning the basics of how to call SIAPI from LotusScript will you be able to write your own advanced OmniFind Notes search application.
Calling SIAPI from LotusScript
When the user enters a query expression on the search form and presses the search button the RunSearch LotusScript library receives control. The RunSearch function is responsible for calling SIAPI to issue the search and process the results.
Figure 4. Editing LotusScript with Domino Designer
It is assumed that the reader is already familiar with LotusScript and knows how to use Domino Designer to modify LotusScript. Figure 4 illustrates the OmniFind.nsf design opened in the Domino Designer. The forms category is expanded showing our two forms named search and settings respectively. The Shared Code/Script Libraries category is also expanded showing two functions, namely ReplaceSubstring and RunSearch. The ReplaceSubstring function is a utility function that will not be described in the article but can be viewed at your leisure. Of more value is the RunSearch function which is shown in the right hand panel and will be discussed in more detail as follows:
The RunSearch LotusScript function
Most of the variables used in RunSearch are defined at the beginning of the function.
With the exception of the SiapiResults defined as a Variant the following SIAPI objects are defined as either JavaClass, JavaMethod, or JavaObjects conforming to the LotusScript-to-Java interface.
Listing 2. SIAPI object definitions
Dim SiapiImplClass As JavaClass
Dim SiapiFactoryObject As JavaObject
Dim SiapiImplMethod As JavaMethod
Dim SiapiAppInfoObject As JavaObject
Dim SiapiSearchServiceObject As JavaObject
Dim SiapiSearchableObject As JavaObject
Dim SiapiQueryObject As JavaObject
Dim SiapiResultSetObject As JavaObject
Dim SiapiResultObject As JavaObject
Dim SiapiResults As Variant
|
There are a few native Java objects that will be needed as follows:
Listing 3. Java object definitions
Dim JavaPropertiesClass As JavaClass
Dim JavaPropertiesObject As JavaObject
Dim JavaPropertiesMethod As JavaMethod
Dim JavaDate As JavaObject
|
The rest of the variable definitions are in support of Notes document operations and general string variables.
Procedurally, the first step is to read the settings document to get the current OmniFind connection information. This information will be later used to create a Java Properties object. Note that this logic could be performed once during initialization.
Listing 4. Retrieving OmniFind settings
Set curdb = notesSession.CurrentDatabase
Set view=curdb.GetView("Settings")
Set doc = view.GetDocumentByKey("Settings", True)
ApplicationID = doc.GetItemValue("ApplicationID") (0)
User = doc.GetItemValue("User") (0)
Password = doc.GetItemValue("Password") (0)
Host = doc.GetItemValue("Host") (0)
Port = doc.GetItemValue("Port") (0)
CollectionID = doc.GetItemValue("CollectionID") (0)
|
The next statement is very important and is responsible for establishing a Java session between LotusScript and the JVM.
Listing 5. Establishing a Java session
Set javaSession = New JavaSession()
|
Similar to the SIAPI Java example, the first SIAPI step is to get the factory object for the OmniFind implementation. In order to do this you must first get the SiapiSearchImpl class from the Java session, get the SearchFactory method from that class, and then invoke the method to obtain the actual factory object as shown below:
Listing 6. Obtaining the OmniFind factory object
Set SiapiImplClass = javaSession.GetClass("com/ibm/siapi/search/SiapiSearchImpl")
Set SiapiImplMethod = SiapiImplClass.GetMethod
("createSearchFactory","(Ljava/lang/String ;)
Lcom/ibm/siapi/search/SearchFactory;")
Set SiapiFactoryObject = SiapiImplMethod.invoke
(,"com.ibm.es.api.search.RemoteSearchFactory")
|
Once you have the factory object, you can create an OmniFind Application Info object which will control which collections this application will be able to search. You should obtain the appropriate Application Id from your OmniFind administrator. The default value is "Default".
Listing 7. Obtaining an application info object
Set SiapiAppInfoObject = SiapiFactoryObject.createApplicationInfo(ApplicationID)
|
Next you need to create a Java Properties object and initialize it with the appropriate OmniFind connection information obtained from the Settings document. Similar to the steps used to get the Factory object, you must ask the Java session for the class and obtain the setProperty method from that class. The username and password properties are only necessary if the OmniFind server has global security turned on. Below are the necessary calls:
Listing 8. Creating a Java properties object
Set JavaPropertiesClass = javaSession.GetClass("java/util/Properties")
Set JavaPropertiesObject = JavaPropertiesClass.CreateObject()
Set JavaPropertiesMethod = JavaPropertiesClass.GetMethod
("setProperty","(Ljava/lang/String;Ljava/lang/String;)Ljava/lang/Object;")
Call JavaPropertiesMethod.invoke(JavaPropertiesObject,"hostname",Host)
Call JavaPropertiesMethod.invoke(JavaPropertiesObject,"port",Port)
Call JavaPropertiesMethod.invoke(JavaPropertiesObject,"timeout","60")
Call JavaPropertiesMethod.invoke(JavaPropertiesObject,"username",User)
Call JavaPropertiesMethod.invoke(JavaPropertiesObject,"password",Password)
|
Now obtain a search service to the OmniFind server using the connection properties defined above and get a searchable object to a specific collection to be searched as allowed by your assigned Application Id.
Listing 9. Obtaining an OmniFind searchable object
Set SiapiSearchServiceObject = SiapiFactoryObject.getSearchService(JavaPropertiesObject)
Set SiapiSearchableObject = SiapiSearchServiceObject.getSearchable
(SiapiAppInfoObject,CollectionID)
|
We are almost ready to issue a search. But before we do we must build a SIAPI Query object from the search string and other parameters provided by the user. I use standard Notes document methods to obtain this information from the current search form. Below shows the creation of the SIAPI query object and the setting of its parameters.
Listing 10. Creating a SIAPI query object
Set SiapiQueryObject = SiapiFactoryObject.createQuery(searchExpression)
Call SiapiQueryObject.setQueryLanguage("en_US")
resultStart = (pgSize * pageNum) - pgSize
Call SiapiQueryObject.setRequestedResultRange(resultStart,pgSize)
|
OmniFind allows you to obtain your results one page at a time. The page size is defined by you. Which page you want is indicated by its starting result number. For example, with a page size of 10 I can request page three with a result start of 20 (0-9=page one, 10-19=page two, 20-29=page three, etc.).
The search method on the searchable object executes the search and returns a SIAPI result set object as shown below:
Listing 11. Issuing a search
Set SiapiResultSetObject = SiapiSearchableObject.search(SiapiQueryObject)
|
The following code is in the RunSearch function but is shown out of context to demonstrate how one would walk the results and extract appropriate information about each result using SIAPI. Once you have extracted the result information into standard LotusScript String variables, you should be able to format and display the search results using standard Notes functionality that you are already familiar with.
Listing 12. Processing search results
numResults = SiapiResultSetObject.getAvailableNumberOfResults()
SiapiResults = SiapiResultSetObject.getResults()
Forall vResult In SiapiResults
Set SiapiResultObject = vResult
resultScore = SiapiResultObject.getScore()
resultTitle = SiapiResultObject.getTitle()
resultDesc = SiapiResultObject.getDescription()
resultURL = SiapiResultObject.getDocumentID()
resultSource = SiapiResultObject.getDocumentSource()
Set JavaDate = SiapiResultObject.getDate()
resultDate = JavaDate.toLocaleString()
End Forall
|
Please refer to the OmniFind Programming Guide for additional information on what methods are available for a SIAPI Result object.
The remainder of this article briefly describes the technique used in the RunSearch function to display the search results as HTML. Results formatted as HTML are not dependent on SIAPI or OmniFind but rather a display choice. It is felt that HTML offers more flexibility in presentation (for example, highlighting query terms in dynamic summaries) and more closely resembles what is offered in the supported OmniFind sample Web application.
The approach is to basically build a string containing the entire HTML to be displayed. Once built, the string is then written out as a Notes stream to the "results" field in the search form. The actual stream is written out as a multipart mime document with the mime type set to text and the encoding set to none.
The first part of the mime document contains the actual HTML string. The next two parts of the mime document contain the GIF images for the icons referenced by the HTML. The actual GIF images are defined in the Initialize section of the RunSearch function.
This article has demonstrated how to add OmniFind search functionality to a conventional Notes application accessed by the Notes client. More specifically, I have shown how to call IBM's Java Search and Indexing API (SIAPI) from within LotusScript. In addition to displaying search results as conventional text fields, this article will also has shown how search results can be displayed as HTML from within the Notes client greatly improving the look and feel of the end user search interface.
With this basic knowledge you can now enhance your Notes applications with OmniFind search functionality beyond what is provided by Domino today. The advantages in using OmniFind are:
- OmniFind can extend your reach outside of Domino to include content from the Web, file systems, relational databases, and other mail and document management systems.
- Your end user's search experience can be enriched with OmniFind's advanced search features
- On the backend, OmniFind can improve the performance of your Domino servers by off-loading full text indexing and search operations to OmniFind.
- And, OmniFind provides true enterprise scale, supporting up to 20 million documents.
| Description | Name | Size | Download method |
|---|---|---|---|
| Sample OmniFind Notes Application | OmniFind.nsf | 1.42MB | HTTP |
Information about download methods
Learn
-
"Discover concepts with Easy Semantic Search"
(developerWorks, Apr 2007): Get an introduction to "Easy Semantic Search" and follow a simple example to see how to implement this functionality.
-
IBM OmniFind page on developerWorks:
Link to articles and tutorials and get the resources you need to advance your skills on IBM OmniFind.
-
Enterprise Search:
Learn more about how to solve your enterprise search and information access needs.
-
IBM OmniFind Enterprise Edition V8.4: Programming Guide and API Reference for Enterprise Search (SC18-9284-03):
Your reference on how to use the Java APIs provided with enterprise search.
- Browse the
technology bookstore
for books on these and other technical topics.
Get products and technologies
- Download
IBM OmniFind Yahoo! edition
to try out a free version of OmniFind.
- Download
IBM product evaluation versions
and get your hands on application development tools and middleware products from
DB2®, Lotus®, Rational®, Tivoli®, and
WebSphere®.
Discuss
- Participate in the discussion forum.
- Check out
developerWorks
blogs and
get involved in the
developerWorks community.
Mr. Leyba is currently serving as an evangelist for Discovery and Search Analytics in IBM's Information Management Division. He is a key spokesperson, responsible for engaging with customers, partners, and developers to articulate IBM's Discovery and Search strategy. In this role, he is also responsible for incorporating customer and developer feedback as well as market trends into IBM's future product direction. Mr. Leyba's expertise lies in the architecture of full text search and retrieval systems and their application in business. He has previously worked on a variety of search related projects including: IBM's WebSphere Enterprise Search product (OmniFind), designed to provide superior performance, scale, and result quality with a broad range of data source support.
Table of contents
Next steps from IBM
My developerWorks community
Tags
Use the slider bar to see more or fewer tags.
Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).
My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
