The Full-Text Search feature in IBM Rational ClearQuest, Version 7.1: Part 2. Install the components

This article guides IBM Rational ClearQuest administrators in installing the components of the Full-Text Search feature introduced in Version 7.1. It is Part 2 of a five-part series about getting started with this feature.

Share:

George Aroush (aroush@us.ibm.com), Advisory Software Engineer, IBM

author photoGeorge Aroush has been with IBM's Rational team for 5 years. He has lead and worked on several projects focusing on ClearQuest. Recently, he has lead, architected, and implemented the Full-Text Search feature for ClearQuest V7.1. Prior to IBM, Mr. Aroush spent 12 years working in the information retrieval, knowledge management, and data mining field. He was responsible for the design and implementation of several search engines and high-performance solutions still in use today. During his free time, Mr. Aroush is an active open source contributor. He leads the Apache Lucene.Net project, which he ported from Java to C#. Mr. Aroush holds a Masters degree in Computer Science from Northeastern University and a BFA from Tufts University.



03 September 2009

Note:
This article is based on using ClearQuest Version 7.1.0. Future versions may directly refer to this article, although the technology is subject to change.

When installing Rational ClearQuest 7.1.0, you have the option to also install Full-Text Search components. If you select it, all the elements for Full-Text Search are installed initially, thereby permitting indexing of a single ClearQuest user database (support for additional databases requires additional configuration). By default, ClearQuest Full-Text Search is not enabled until it is customized and configured for your database. Work with your schema designer to decide which record types and fields to enable for Full-Text Search. This process requires careful thought and is bound to your schema and your business needs.

Components overview

The diagram in Figure 1 shows a high-level architectural view of key components that make up ClearQuest Full-Text Search.

Figure 1. Key components of the Full-Text Search feature
Web Application Server Diagram

Solr application (search server)

The underlying Full-Text Search engine used by ClearQuest 7.1 Full-Text Search is called Solr. It sits on top of Lucene and provides easy access to the Lucene search engine for configuring, indexing, searching, and administering. To learn more, see the Apache Solr and Apache Lucene Web sites.

Solr schema file

This is a Solr file, and it is always called schema.xml. As an administrator of ClearQuest Full-Text Search, you will be editing this file to specify which fields in your ClearQuest Database you will index and search on in addition to specifying the language of your database (if it is other than English).

There are additional advanced options available for you to set. You can learn more about them by visiting the Solr Web site.


Lucene is the backend of ClearQuest Full-Text Search. Solr uses Lucene for searching and indexing ClearQuest records. You will not have direct access to Lucene; you will interact with it through Solr and ClearQuest.

Lucene index file

A Lucene index file consists of several files on a file system. As you add, modify, or delete ClearQuest records, the Lucene index is updated. You do not need direct access or interaction with the Lucene index files.


ClearQuest Record Extractor

The Record Extractor (cqtsdbcrawler.pl and cqsearchprofile) is installed under WebSphere Application Server as a component and under the ClearQuest directory as a command line tool component. While the bits of those two installed components are identical, their usage and end goal is different. The Record Extractor under WebSphere Application Server provides continuous updating of the Lucene index with those of ClearQuest records. That is, as new records are added to Rational ClearQuest, modified or deleted, the WebSphere Application Server Record Extractor sends those records to Solr for indexing by Lucene at set batch intervals. After Lucene indexes a ClearQuest record, it becomes immediately available for searching from the ClearQuest Web 7.1 Full-Text Search interface. This mode of record extraction is called Update Mode Record Extractor.

The command line Record Extractor under the ClearQuest directory provides batch extraction of ClearQuest records. The Record Extractor must be started by the Administrator, usually once, to extract existing records from ClearQuest and send them to Solr for their initial indexing by Lucene. After indexing is completed, existing ClearQuest records become available for searching from the ClearQuest Web 7.1 Full-Text Search interface. This mode of record extraction is called Batch Mode Record Extractor.

The third way to run the command line Record Extractor is to re-index any ClearQuest record that the update or batch mode failed to process. This mode of record extraction is called Repair Mode Record Extractor.

All modes of Record Extractor use the same ClearQuest Search Property file and configuration.


ClearQuest Search Property file

This is an XML file used by the Record Extractor and ClearQuest core (when Full-Text Search is enabled). This file contains settings specific to your ClearQuest schema, such as the location of Solr, the repository, and the record types and fields to index and search. The Batch and Update Mode Record Extractors both use the file system instance of the ClearQuest Search Properties XML file, while ClearQuest core uses the ClearQuest user database instance of the ClearQuest Search Properties XML file as stored within the TEXT_SEARCH_PROPERTY master property.


Next steps

Part 3 of this five-part series shows you how to enable configure and enable the Full-Text Search feature (see the link to “More in this series”).


Acknowledgement

Special thanks to David Sampson, a staff technical support engineer in IBM Rational Client Support, who serves on the ClearQuest Full-Text Search Cross-Functional Team.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=424211
ArticleTitle=The Full-Text Search feature in IBM Rational ClearQuest, Version 7.1: Part 2. Install the components
publish-date=09032009