Useful Development and Debugging Tools

Determining the exact interaction between your browser and the server can be complicated by factors such as POST requests, CGI parameters generated by JavaScript, filtering based on HTTP headers, and cookie-based logins. It is very useful to use a tool to monitor your browser's interaction with the server. Similarly, when developing your search applications, the complexity of the code for some Web sites can make it difficult to determine the correct nesting of HTML elements that you must identify when manually developing parsers or when writing the XPath expressions that refer to specific portions of the content that you are trying to extract. Tools that can analyze and display the nesting of complex portions of the Web sites that you are searching can simplify and expedite your development efforts.

Though enterprise search applications are typically designed to be browser-independent, using the Firefox browser during your development efforts enables you to take advantage of a large number of excellent (and free) diagnostic and analytic add-ons that are available for this browser. If you are creating complex meta-search connectors with Watson™ Explorer Engine, we strongly suggest that you download and install the latest version of Firefox from the Mozilla Foundation for use in your development efforts. You will still need to test your search applications in other browsers used by your user community, such as Microsoft's Internet Explorer and Apple's Safari, but Firefox and its freely-available add-ons can be very useful while you are designing and debugging search applications based on Watson Explorer Engine.

When installing Firefox, select its Custom installation mechanism, and install the DOM Inspector. As the name suggests, this tool enables you to inspect the Document Object Model (DOM) tree of HTML and XML-based documents in your browser by selecting any element from a tree-structured view of your documents.

Once you have installed Firefox, the following Firefox add-ons can be very useful in your development efforts:

Firebug - This extension enables you to edit, debug, and monitor CSS, HTML, and JavaScript in any web page. When developing Watson Explorer Engine-based search applications, Firebug makes it easy to view the HTML source code for web pages in order to derive optimal XPath expressions for specific content elements.
warning: Firebug can cause significant performance problems when used as part of your default Firefox environment. We suggest installing Firebug but disabling it during normal use. (You can disable an add-on from the Tools > Add-ons dialog in Firefox. Select the Firebug extension and click Disable to disable this extension. You will have to restart Firefox before Firebug is actually disabled.) You can then selectively enable the Firebug extension when you are working on or debugging CSS, HTML, or Javascript.
It's All Text! - This extension adds an edit button below all editable text areas on a Web page (and therefore in the Watson Explorer Engine administration tool), enabling you to edit text areas using your favorite external text editor.
Live HTTP Headers - This extension enables you to view HTTP Headers as you request pages. This can be very useful when you are connecting to sites that use POST requests, redirects, or similar functionality.
Tamper Data - This extension enables you to view and modify HTTP/HTTPS headers and POST parameters, and to trace and time HTTP requests and responses.
Note: This extension has problems with certain versions of the Google Web Accelerator, and may cause your browser to crash if you are using that tool.
Web Developer Toolbar - This extension provides a toolbar and rich set of menus that enable you to view and edit almost any aspect of a web page. Among the most interesting tools provided by this extension are the CSS tools to help design your application's result pages and the Form tools to convert forms from POST to GET.
Tip: One especially useful feature provided by the Web Developer Toolbar is the ability to temporarily disable browser caching. This can be invaluable during development when you want to ensure that the results of a search are always up-to-date, rather than potentially displaying cached results from a previous but identical search. If you are actually displaying the Web Developer Toolbar, disabling caching is done from the Disable menu, which is the left-most item on the toolbar. If you are not displaying the toolbar but have loaded the Web Developer Toolbar extension, you can temporarily disable caching by selecting the Tools > Web Developer > Disable > Disable Cache menu item. Caching is disabled when a checkmark is displayed beside this menu item. To re-enable caching, re-select this menu or toolbar item.
XPath Checker - This extension enables you to right-click on any portion of a search result and select View XPath from the pop-up contextual menu to see the XPath for that element. You can also edit the XPath expression that is currently displayed, which provides visual feedback as your modifications sellect different elements or attributes.
XPather - This extension, which is similar to the XPath Checker, provides a more detailed view of the XPath information for the element that you right-click on.

To see a complete list of available Firefox add-ons and search for others that might be useful in your development efforts, see the Mozilla Firefox Add-Ons Site.