Step 3. Transform legacy data to GJXDM type XML document
A major goal at the outset of this project was to find the right set of IBM tools that could transform legacy data into the IncidentReport XML document. The XML schema used in the IncidentReport XML is based on the standard GJXDM, Global Justice XML Data Model, schema. We looked at the XML tooling in the following IBM products:
- WebSphere Rational Application Developer V 6
- WebSphere Rational Application Developer V 188.8.131.52
- DataStage v8.0 XML Pack
- WebSphere Transformation Extender for DataStage V 184.108.40.206
- WebSphere Transformation Extender Designer Studio V 220.127.116.11
Only the WebSphere Transformation Extender(WTX) Designer Studio V 18.104.22.168 successfully generates an XML tranformation map for the IncidentReport schema. However, InfoSphere DataStage requires the WTX for DataStage product. The problem is that WTX for DataStage v 22.214.171.124 contains an older version of the WTX engine which can not generate the XML transformation map for the GJXDM schema successfully.
We worked around the problem by using both products. We used the WTX Designer Studio v126.96.36.199 to generate the XML transformation map for the GJXDM schema. We used the WTX for DataStage v188.8.131.52 to execute the map as part of a DataStage job. We installed WTX for DataStage v184.108.40.206 on the workstation where the DataStage Designer Client was installed. We installed WTX Designer Studio v220.127.116.11 on a different workstation since the installation of WTX for DataStage installs a copy of the WTX Designer Studio as well as the WTX-DataStage integration code.
Essentially, a WTX transformation map is created by mapping a source WTX type tree to a target WTX type tree. There are different methods of creating a WTX type tree. One method is to import an existing XML schema; this is the method described in this tutorial. Table 3 lists the steps you follow to create the transformation map:
Table 3. Steps to create the WTX transformation map
|1||Analyze the data to determine mapping.|
|2||Create a simple XML schema for the legacy data. We named it myInternalIncRpt.xsd.|
|3||Create a WTX type tree map for the XML IncidentRepor.xsd schema.|
|4||Create a WTX type tree map for myInternalIncRpt.xsd schema.|
|5||Create a WTX Transformation Map that maps the myInternalIncRpt type tree to the IncidentReport type tree.|
Analyze the data to determine which legacy data fields to use to fulfill the IncidentReport request, and determine how each field will be mapped to the IncidentReport schema. See Downloads for an Excel spreadsheet, DBXml2IRMap.xls, in the sample code that lists the data fields from the legacy data sources that were selected to fulfill the IncidentReport. The spreadsheet also shows the mapping from the legacy data field to the corresponding data field in the IncidentReport schema.
Multiple rows of legacy data might be required to fulfill a request for an IncidentReport document. Currently, the WebSphere TX Map Stage only accepts one input card (WTX terminology for an input data stream). Because of this, we decided to transform the legacy data into a simple XML document. Doing this multiple rows of data into one XML document and feeds that XML document into the WebSphere TX Map Stage as one input card.
To get a copy of the XML schema file, myInternalIncRptNoNS.xsd, please go to Downloads.
Besides using the myInternalIncRptNoNS.xsd to generate the WTX type tree, the myInternalIncRptNoNS.xsd schema was also used in the DataStage job to transform the legacy data in a relational format to an XML document. This is discussed further in a later section, "Create the DataStage job".
The IncidentReport XML schema is publicly available. It is included in the Downloads package. Creating the WTX type tree is a matter of importing the IncidentReport XML schema into the WTX Type Designer.
- Select Start -> All Programs -> WebSphere Transformation
Extender v8.1 -> Design Studio -> Type Designer.
Figure 9. Select WTX type designer
- From the Type Designer menu bar, select Tree -> Import.
From the Import window, select XML Schema and click
Next. Figure 10 shows these steps:
Figure 10. Import XML Schema
- From the Import Wizard - XML Schema window, enter the location of
the XML Schema and click Next.
Figure 11. Enter location of the XML schema file
- Select the language and click Next as Figure 12 shows:
Figure 12. Select language
- Enter the type tree name. Under Validation, select Xerces.
Figure 13. Select validation type
- Once the Type Designer completes generating the type tree, click
Finish as Figure 14 shows:
Figure 14. Import of XML schema completed
- After the import step is completed, you need to analyze the newly
generated type tree for structural and logical correctness. From
the Type Designer menu bar, select Tree -> Analyze ->
Structure Only. A bunch of warnings will result. They are
harmless. You can ignore them.
Figure 15. Analyze the type tree structure
- Select Tree -> Analyze -> Logic Only. There should
be no error. If there is any error, you need to resolve the error
before type tree can be used for creating the map. Save the newly
generated type tree.
Figure 16. Analyze the type tree logic
Follow the same steps listed above to create a WTX type tree for the legacy data XML schema, myInternalIncRptNoNS.xsd.
After the WTX type trees for the IncidentReport XML schema and the legacy data XML schema are created, the next step is to create the WTX transformation map.
It is beyond the scope of this tutorial to provide detailed steps on how to use the WTX Map Designer or how to map a source type tree to the target type tree. However, our source code is available for download. If you open the IRMap.mms in your Map Designer, you will see how the legacy data type tree is mapped to the IncidentReport type tree. Refer to the Resources section for a list of suggestions on where to get additional learning modules for the WTX product.
- We found it helpful to place all the XML schemas, WTX type trees and WTX maps into the same subdirectory. Doing it this way keeps all the files together, and we liked copying the entire directory into the DataStage Server subdirectory. This is discussed in greater detail in a later section, Create the DataStage job.
- Generate a sample XML document from the legacy data XML schema. For example, we generated the myInternalIncRptNoNS.xml from the myInternalIncRptNoNS.xsd file. Use the generated XML document to test out the WTX Map.
- There was a known problem in the XML Pack of DataStage v8.0.1 that it could not correctly process XML namespaces. Therefore, the myInternalIncRptNoNS.xsd had to be created without any namespace information.