Developing a Parser

Parsers are used in conjunction with a transport Connector to interpret or generate the content that travels over the Connector's byte stream. However, sometimes you may want to parse data that is presented in a very specific format; for this purpose you will need to implement your own Parser.

Implementing the Parser Java source code

All Tivoli® Directory Integrator Parsers implement the com.ibm.di.parser.ParserInterface Java interface. This interface provides a number of methods to implement that are common to all parsers. Usually the parsers that you write will not require implementing all methods provided by the interface but only a subset of them. For this purpose you can use the com.ibm.di.parser.ParserImpl abstract class that implements the ParserInterface. The ParserImpl class contains the core Parser functionality so you can subclass it when implementing your own Parser.

There are two types of parsers: ones that read from a stream and return an Entry; and others that take an Entry and write it to a stream.

Once the Parser is constructed you have to configure it. This includes setting the input/output streams and configuring some additional parameters if needed. This is usually made by the hosting component (for example, a Connector). When finished with this job, the next step is initialization of the Parser where resources for future needs are allocated and any other initialization takes place. Generally the hosting component takes care of both configuring and calling the initialization method of the Parser. Next comes the most significant moment of using the Parser - writing or reading the entries. This is the place where the actual parsing happens. Finally, when the Connector has finished transporting the entries, the Parser must be closed. When closed, the Parser releases the resources that were used in the previous stages as well as closing the input and output streams.

For an example of a Parser implementation, look at the ExampleParser.java Parser included in Tivoli Directory Integrator. These are some of the important methods you will usually need to implement:

public void setInputStream(InputStream is)

Set the input stream attribute of the Parser here. This method is overloaded and can take "String" and "Reader" as arguments, too. The abstract class com.ibm.di.parser.ParserImpl provides implementations for the three methods that set the input stream of the Parser. If you subclass com.ibm.di.parser.ParserImpl you can override some of these methods or leave the default implementation of the super class. However, if you implement the interface you will have to provide implementation for all of them.

Note that when you open the input stream, it is your responsibility to close it. This is usually done in the closeParser() method. The com.ibm.di.parser.ParserImpl abstract class provides default implementation for closing the Parser input and output streams.

public void setOutputStream(OutputStream os)

Set the output stream of the Parser here. The output stream is passed as an argument. This method has an overloaded version that takes a "Writer" object as an argument. Like "setInputStream(...)" the abstract class com.ibm.di.parser.ParserImpl provides implementation for both methods that sets the output stream of the Parser. If you subclass com.ibm.di.parser.ParserImpl you can override some of these methods or leave the default implementation of the super class. However, if you implement the interface you will have to provide implementation for all of them.

Note that when you open the output stream, it is your responsibility to close it. This is usually done in the closeParser() method. The com.ibm.di.parser.ParserImpl abstract class provides a default implementation for closing the Parser input and output streams.

public void initParser()

Put any initialization code here. This method is usually called by the hosting component (for example, a Connector) to initialize the Parser.

You can allocate resources you may need in future, as well as setting any parameters or additional chained parsers. This method may not be required for all implemented parsers.

Here is an example of how you can access parameters. This set of code is part of the included example "ExampleParser.java".

str = getParam("attributeName");
  		if (str != null && str.trim().length() != 0) {
  			attrName = str;
  		}  

Note that this method is called after setting of input and output streams is done.

public Entry readEntry()

This method is similar to the "getNextEntry()" method used to implement a Connector. It returns the next entry from the current input stream. In case that the input stream is depleted a null value is expected to be returned. This is the place where the actual parsing takes place.

Make sure you have initialized the input stream properly. In order to set the input stream you can use the setInputStream(...) method. You can use the getReader() method to get the reader object.

Generally input streams are initialized by the hosting component (for example, a Connector).

public void writeEntry(Entry entry)

Use this method to write an entry using the current output stream. The entry that will be written is passed as an argument. This is the place where you have to parse the data of the entry and write it in the proper format to the output stream.

In order to get the writer you can use the getWriter() method which returns a "java.io.BufferedWriter".

Generally the output stream is initialized by the hosting component (for example, a Connector).

public void flush()

This method is usually called by some hosting components to flush any in-memory data to the current output stream. So, to make sure that all the data is written and there is nothing left in the memory call this method.

public void closeParser()

This method is called by the hosting component (for example, a Connector) to close and release the Parser resources. Use this method to close and release any resources that may no longer be used and when the Parser will not be invoked anymore. In most cases this would be the input and output stream but if any additional resources were used, they should be released as well.

The com.ibm.di.parser.ParserImpl abstract class provides an implementation for this method but if you implement the interface you will have to write it by yourself.

Building the Parser source code

When building the source code of your Parser, include in your CLASSPATH the jar files from the "jars" folder of the IBM® Tivoli Directory Integrator installation. As a minimum, you would need to include "miserver.jar" and "miconfig.jar". Keep in mind that the source code must be compiled for Java 5 or older.

Implementing the Parser GUI configuration form

The Parser GUI is implemented in the same way as for a Connector. Use a "tdi.xml" file to describe the Parser configuration form by using the same syntax as used for Connectors.

Packaging and deploying the Parser

Packaging and deploying a Parser is just like packaging and deploying a Connector:

You need a jar file that contains:

1. The class file(s) of the Parser
2. "tdi.xml" file placed in the root of the jar file. Note that this file is used to register the Parser. Without it the code will not be loaded, and you will not be able to use the Parser in the Tivoli Directory Integrator Configuration Editor (however you will be able to call it from scripts).

After you have created the jar file of the new Parser, you only need to drop that jar file in the "jars" folder in the Tivoli Directory Integrator installation. You can create your own folder and put the jar there but the general place where parsers are stored is the "jars/parsers" folder. The next time the Tivoli Directory Integrator is started it will automatically load the new Parser and it will be ready for use.