Java interfaces for splitters and annotators

The Java™ interfaces are included with the IBM® Operations Analytics - Log Analysis are described here.

The implementation process for the Java-based splitters and annotators is:
  1. Create Java classes that implement specific interfaces. You create one class to implement the splitter interface and you create one class to implement the annotator interface. The JAR file that contains the classes for each of these interfaces is installed with IBM Operations Analytics - Log Analysis.
  2. Import the interface jar files into the Insight Pack project under the lib directory. The name of the JAR files required for compiling are unity-data-ingestion.jar and JSON4J.jar. After successful compilation, the Java splitter and annotator implementation class files are packaged in a JAR file which is included within the Insight Pack when it is exported from the tooling.
  3. Use the pkg_mgmt script utility to install the Insight Pack into the IBM Operations Analytics server. During the installation, the pkg_mgmt utility copies the implementation JAR to the required location in the IBM Operations Analytics server.
.

Splitter interface

The Java splitter interface is defined as follows:

 package com.ibm.tivoli.unity.splitterannotator.splitter;

 /************************************************************************     
  * This interface defines the APIs for Java based Splitters and is used    
  * by third party custom Java Splitter developers
  *  
***********************************************************************/
public interface IJavaSplitter
{
/******************************************************************
	 * Split a batch of log records packaged in the input JSON
	 * 
	 * @param batch
	 * @return
	 * @throws JavaSplitterException
	 ******************************************************************/
 public ArrayList<JSONObject> split( JSONObject batch ) throws Exception ;
   
/*****************************************************************
	 * Data section
	 * ***************************************************************/
	  public static final String IBM_COPYRIGHT = 
	      "Licensed Materials - Property of IBM\n" 
	    + "LK3T-3580\n" 
	    + "(C)Copyright IBM Corporation 2002.\n" 
	    + "All Rights Reserved.\n" 
	    + "US Government Users Restricted Rights - Use, duplication \n" 
	    + "or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.\n\n";
Input JSON

The input JSON is primarily a batch of raw log records that needs to be split into logical log records according to a particular criteria (for example, timestamp). The class implementing the IJavaSplitter interface provides the logic that performs the splitting for the given criteria.

The basic structure of the incoming JSON object is:

{
	   “content”:	{
                           “text” :      // raw text to be split
		           ,
                           ,
			},

           “metadata”:  {
                           ...meta data fields, eg. hostname, logpath,
                           other fields passed from client...
			}
 }
Output JSON

The class implementing IJavaSplitter must return an ArrayList of JSONObjects . Each JSONObject represents either a complete logical log record or a partial log record (for cases where the splitter was unable to specifically determine that the record was complete) and meta-data to indicate whether the included record is complete or not. 

Output JSON:
     {
         “content”:	{
                       “text” : // text for this complete/partial log record
                       ,
                       ,
                   },
         “metadata”:	{
                       “type”:  , // “A” = complete log record
                                  // “B” = partial log record at end
                                  // “C” = partial log record at beginning 
                                    ,
                        }
         “annotations”:	{
                         "timestamp": // include the timestamp for the
                                         current record represented in
                                         this JSON object
                        
                        }
     }

Annotator interface

The Java annotator interface is defined as follows:

 package com.ibm.tivoli.unity.splitterannotator.annotator;

  /************************************************************************
     * This interface defines the APIs for Java based Annotators and is used
     * by third party custom Java Annotator developers
     * 
     ***********************************************************************/
    public interface IJavaAnnotator
   {	
	/*****************************************************************
	 * Annotate the input log record & return the output with annotations
	 * 
	 * @param input
	 * @return
	 * @throws JavaAnnotatorException
	 *****************************************************************/
	public JSONObject annotate( JSONObject input ) throws Exception ;
	
	/*****************************************************************
	 * Data section
	 * ***************************************************************/
	public static final String IBM_COPYRIGHT = 
	    "Licensed Materials - Property of IBM\n" 
	    + "LK3T-3580\n" 
	    + "(C)Copyright IBM Corporation 2002.\n" 
	    + "All Rights Reserved.\n" 
	    + "US Government Users Restricted Rights - Use, duplication \n" 
	    + "or disclosure restricted by GSA ADP Schedule Contract with
         IBM Corp.\n\n";
   }
Input JSON

The input JSON includes a logical log record (formed by splitter or raw record if no split was performed) that is now ready for annotation. The class implementing the IJavaAnnotator interface provides the logic that performs the annotation against the given input record and creates an output JSONObject representing the JSON structure containing the annotations.

The basic structure of the incoming JSON object is:

{
	   “content”:	{
                   “text” :      // logical record to be annotated
			},

        “metadata”:  {
                       ...meta data fields, eg. hostname, logpath,
                          other fields passed from client...
			}
}
Output JSON

The class implementing IJavaAnnotator must return a single JSONObject representing a JSON data structure containing the original data passed as input plus the annotated fields parsed from the incoming record. The following sample JSON structure depicts the format of the data that is expected to be returned in the object.

Output JSON:
   {
     “content”:	{
                  “text” : // same text as passed in the input JSON object
                 },
     “metadata”: {
                     ...meta data fields, eg. hostname, logpath,
                     other fields passed from client...
		},

       “annotations”:	{
                         ...annotation fields and their values produced by
                            IJavaAnnotator implementation
                     }
   }