With Google’s current and ever expanding index of over 3 billion Web pages, it is easy to understand why it is one of the best tools to conduct industrial-strength research on the Internet. In addition to its quantity of Web pages, the quality of the search results is high due to its proprietary search algorithm that is based on relevancy and popularity. Recent reports indicate that Google answers 200 million search requests daily (an average of 2,300 per second!) in 88 different languages. And according to SearchEngineWatch.com, Google handles 75 percent of all Web-based search queries.

Obviously, many Web users are familiar with Google and its search features. What may not be as well known is the fact that you can incorporate Google's search functionality into your own Domino applications using Google's Web API service. (You can download this API free as we explain in the next section.) This article describes how to add Google's search functionality to a Domino application. We briefly introduce the Google APIs and how they work. Then we examine a sample application that integrates Google search functionality with a Domino application. We conclude by offering some ideas how you can expand and customize our simple example. (All examples shown in this article are contained in a Domino database you can download from the Sandbox.)

This article assumes that you're an experienced Domino developer familiar with writing Domino agents. And although advanced Java experience is not required to understand our example, a basic knowledge of Java is highly recommended.

The Google APIs

To use Google’s APIs, download the Google Web APIs developer’s kit (googleapi.zip) from Google's Web APIs download page and create a free Google account for a license key. The key allows you a maximum of 1,000 searches daily. (You must include this license key in your query when making the search request. You do this programmatically, so users don't have to type in the code each time they perform a query.) The developer’s kit comes with documentation and examples that give you a basic understanding of how Google’s automated searches work.

After you download googleapi.zip, unpack it to the c:\googleapi directory. You can then quickly invoke the Web APIs by typing the following command at the MS-DOS prompt: java -cp googleapi.jar com.google.soap.search.GoogleAPIDemo <key> search Lotus Domino > result.html where < key > is your registration key, "Lotus Domino" is the item you want to search for, and result.html is the name of the file into which we want the search results to be written. For instance, the following example:

…
Start Index = 1 
End Index = 10 
Estimated Total Results Number = 663000 
…  

shows the first 10 items in the display results from an estimated 663,000 items.

However, if you try the same query on the Google Web site, you may get a different result. For example, the same query for "Lotus Domino" at Google results in an estimated 596,000 items. (The Google API Support team has confirmed that both the Web APIs and the Google site itself use the same search engine and index pages, but slight differences in results may be caused by the many data centers that Google operates.) A quick look at GoogleAPIDemo.java in the googleapi directory:

 …
GoogleSearch s = new GoogleSearch();
…
GoogleSearchResult r = s.doSearch();
…  

reveals that the doSearch() method invokes the Google search, after the license key and the search query string attributes are set.

Before we move on to our example, let's take a quick look at several methods in Google’s Web API that you will use:

• setKey() sets the user license key.
• setQueryString() sets the query string.
• doSearch() invokes the Google search.
• toString() returns a formatted representation of a Google search result.
• getResultElements() returns an array of result elements.
• getSnippet() returns a snippet that shows the query in context on the URL where it appears.
• getSummary() returns the ODP summary if this document is contained in the ODP directory.
• getTitle() returns the title of the search result, formatted as HTML.
• getURL() returns the absolute URL of the search result.

In addition, the following set of methods can improve the sophistication of your search application:

• setStartResult() sets the index of the first result to be returned. For instance, if there are 500 results, you may want to start at 100.
• setMaxResults() sets the maximum number of results to be returned per query. The maximum value per query is 10. If you perform a query that doesn't have many matches, the actual number of results you get may be smaller than what you request.
• setFilter() enables or disables the "related-queries" filter. This filter eliminates results that are very similar.

Back to top

The simpleSearch agent

Let us begin our example by creating a Notes agent called simpleSearch in a new database named GoogleSe.nsf. You can download this database from the Sandbox. This database contains the simpleSearch agent, as well as the goSearch agent and the forms and view discussed later in this article. We built this application on Windows 2000, but as a Java application it can run on any platform. No special proxy setting is required to run the Google APIs from Lotus Domino. When the APIs are invoked, they use whatever configuration Domino has.

First, copy the following lines of code from the file GoogleAPIDemo.java in the developer’s kit (note that clientKey is initialized by the license key sent to you after you register with Google):

...		
	String clientKey = "4B0KufpQFHJxhAxzua0tR11ElLNrHRJ6";
    	String directive = "search";
    	String directiveArg = "Lotus Domino";

    	// Report the arguments received
    	System.out.println("Parameters:"); 
	System.out.println("Client key = " + clientKey);
	System.out.println("Directive  = " + directive);
	System.out.println("Args       = " + directiveArg);

    	// Create a Google Search object, set our authorization key
    	GoogleSearch s = new GoogleSearch();
    	s.setKey(clientKey);

	// Depending on user input, do search or cache query, 
	 then print out result
 	try {
	      if (directive.equalsIgnoreCase("search")) {
		s.setQueryString(directiveArg);
    		pw.println("about to begin doSearch...");
    		GoogleSearchResult r = s.doSearch();
		pw.println("doSearch ok");
        		System.out.println("Google Search Results:");
		System.out.println("======================");
		System.out.println(r.toString());
	...  

In Domino Designer, create the simpleSearch agent and paste in the preceding code. Then execute the agent. (The search string in this example is initialized to "Lotus Domino.") To view the results of this query, open the Java Debug Console after simpleSearch finishes running. You should see results similar to the following:

Our simpleSearch agent provides a quick way to view your search results, showing the first 10 of the estimated 720,000 items found. For example, the first item contains the following:

 URL  = "http://www.lotus.com/" 
Title = "IBM Lotus software" 
Snippet = " ... and societal demands Domino Web Access demo upgraded to 6.5 
All you've come to expect -- Lotusphere 2004 Basex Excellence Award goes to 
IBM Lotus Workplace ...  "
Directory Category = {SE="", FVN="Top/Computers/Software/Operating_Systems/
 Mainframe/IBM/Software"}
Directory Title = "Lotus Software" 
Summary = "IBM subsidiary best known for multi-platform Notes and Domino 
 messaging groupware  supports z/OS-based..." 
Cached Size = "20k" 
Related information present = true 
Host Name = ""

If you run the agent locally, make sure googleapi.jar is in the Notes program directory and is assigned to JavaUserClasses in your local Notes.ini file. (JavaUserClasses has a limitation of 256 characters, so if adding googleapi.jar to it exceeds this limit, it won't be recognized by Domino.)

Back to top

Adding a Google-like interface

The search results returned by simpleSearch are informative, but are obviously unsophisticated in terms of presentation. However, you can greatly improve the usability of these results by presenting them in a format similar to the one used by Google. To do this, we create a new agent called goSearch, using the code in simpleSearch as the starting point. We also add two forms, Search and Result, to the GoogleSe.nsf database. (The complete GoogleSe.nsf database, containing the forms as well as the goSearch and simpleSearch agents, is available for download from the Sandbox.)

Search and Results forms

First, add a query form (which we call Search) to your GoogleSe.nsf database. This allows users to enter and save their query strings:

Then add an output form named Result to display the results of the queries:

The goSearch agent

Now let's build an agent that offers more sophisticated functionality than the simpleSearch agent, including automation to process a list of queries and a more "Google-like" look and feel. To begin, copy the code in simpleSearch and paste it into goSearch. Then add the following code to have the results title appear in the same text color used by Google and to render HTML in rich text fields in Notes:

... 
style.setColor(RichTextStyle.COLOR_BLUE); 
                                      
//renders HTML into rich text fields                                         
String str = re[a].getTitle(); 
while (str.length() > 0 ) { 
        int i = str.indexOf("<b>"); 
        int j = str.indexOf("<b>"); 
        style.setBold(RichTextStyle.NO); 
        int k = i; 
        int len = 3; 
        if (j<i || (j>i && i==-1)) { 
                style.setBold(RichTextStyle.YES); 
              &n! bsp; k=j; 
                len=4; 
        } 
        //no more <b> or </b> left on str 
        if (i==-1 && j==-1) { 
                title.appendStyle(style); 
                title.appendText(str);                 
                break; 
        } 
  !       String sstr = str.substring(0,k); 
        if (sstr.length()==0) { 
                sstr=""; 
        } 
        title.appendStyle(style); 
        title.appendText(sstr);                 
         str = str.substring(k+len); 
 } 
...  

Next, add the following code to the goSearch agent. This snippet incorporates the query results into the Result document:

 …
Document ndoc = db.createDocument();
ndoc.appendItemValue("Form", "Result");
RichTextItem title = ndoc.createRichTextItem("Title");
...
RichTextItem snippet = ndoc.createRichTextItem("Snippet");
...
ndoc.appendItemValue("URL", re[a].getURL());
...
ndoc.save();
…  

Finally, add the following to execute all query strings saved as Search documents and generate corresponding results as Result documents.

... 
View view = db.getView("AllSearches"); 
Document doc = view.getFirstDocument(); 
while (doc != null) { 
... 
        int a=r.getStartIndex() - 1; 
        int b=r.getEndIndex()  - 1; 
        //reiterate the result set 
        while (a<=b) { 
                ... 
&! nbsp;               a++; 
        } 
doc = view.getNextDocument(doc); 
} 
...   

The preceding example allows for batch queries in Google’s search engine and batch results saved for later analysis. Because results are saved, it also lets you analyze trends by monitoring the frequency with which certain topics are searched. You can do this by sorting and displaying Result documents by query string and date, as shown in the following AllResultsByTitleDate view:

The Title field is a rich text field. Therefore, you need to create the new text field TitleText to store the unformatted text value in Result documents, so you can use it as a column value in the preceding view: ndoc.appendItemValue("TitleText", title.getFormattedText(false, 0, 0));

Back to top

Other ideas to consider

The goSearch agent can be scheduled to run at an off-peak hour to reduce traffic and improve efficiency. Another idea to improve the goSearch agent is to call the parser whenever an HTML formatted string is encountered:

... 
class Parser { 
                        
        //renderHTML takes 5 arguments: HTML string, 
         rich text style, rich text item 
         reference, delimiter 1 and 2 
    void renderHTML(String html, RichTextStyle style, RichTextItem rtitem, 
     String delim1, String delim2) {                         
                try { 
                        System.out.println("html:" + html); 
                                                                        
                              style.setColor(RichTextStyle.COLOR_BLUE); 
                                      
                        //renders HTML into rich text fields                                         
                        while (html.length() > 0 ) { 
                                int i = html.indexOf(delim1); 
                                int j = html.indexOf(delim2); 
                                style.setBold(RichTextStyle.NO); 
                                int k = i; 
                                int len = 3; 
                                if (j<i || (>i && i==-1)) { 
                                        style.setBold(RichTextStyle.YES! ); 
                                        k=j; 
                                        len=4; 
                                } 
                                //no more <b> or </b> left on str 
                                if (i==-1 && j==-1) { 
                                        rtitem.appendStyle(style); 
                                        rtitem.appendText(html);                 
                                        break; 
                                } 
                                String sstr = html.substring(0,k); 
                                if (sstr.length()==0) { 
                                        sstr=""; 
                                } 
                                rtitem.appendStyle(style); 
                                rtitem.appendText(sstr);                 
                                 html = html.substring(k+len); 

                         } 
                                          
                } catch(Exception e) { 
                        e.printStackTrace(); 
                } 
      } 
}  

This results in a more compact agent class.

You can further extend the concept of rendering HTML input strings into Notes rich text fields by expanding the set of HTML tags that this parser can handle (for example, to <h1>…</h1> or <p>) through the use of the delim1 and delim2 parameters.

Back to top

Summary

You can harness the power of Google in Domino to query high volumes of search strings. Their results are stored as documents and, over a period of time, can be used in trend analysis. In addition, you can monitor when new information for specific topics is available as Web pages. Equally important, your Domino applications can now offer Google search functionality to your users. This provides search features with which your users are likely to be very familiar, so they should find these features easy to use. This functionality is also known to be highly stable and reliable. You should consider using Google's API in any Domino application in which fast, large-volume Web searches are a requirement.

Resources

• Download the Google Web APIs developer’s kit (googleapi.zip) from Google's Web APIs download page.
  
  
• Download the sample code used in this article.
  
  
• "Integrating Amazon Web Services with your Domino applications" is the first of a three-part article series explaining how you can use Amazon.com's Web functionality in your Domino applications.
  
  

About the author

Rate this article

Comments

Back to top

Table of contents

• The Google APIs
• The simpleSearch agent
• Adding a Google-like interface
• Other ideas to consider
• Summary
• Resources
• About the author
• Comments

Local resources

• IBM Innovation Center - Austin, TX
• IBM Innovation Center - Chicago, IL
• IBM Innovation Center - Dallas, TX
• IBM Innovation Center - San Mateo, CA
• IBM Innovation Center - Waltham, MA
• Technical Briefings in North America

Dig deeper into Lotus on developerWorks

Discover knowledge paths

Skill-building guides for Linux, open source, cloud, Java, business analytics, and more.

Special offers