Step 4: Create the DataStage job
Figure 17 shows the IncidentReport DataStage job. The job name is IncRptFedViewParm. The job sequence is from left to right. It follows the pattern of data retrieval, data transformation, data mapping and data delivery.
Figure 17. IncidentReport DataStage job
Table 4 gives a brief description of each stage in the job.
Table 4. Description of the stages in the IncRptFedViewParms job
|DB2 API (incReport)||Retrieves data from the IncReport federated view.|
|Transformer||Transforms the data retrieved from the incReport Stage to the metadata defined by the myInternalIncRptNoNS.xsd schema.|
|XML Output (DBXml_Output)||Generates an XML document based on the myInternalIncRptNoNS.xsd schema.|
|WebSphere TX Map||This Stage calls the WebSphere Transformation Extender to map the XML document generated by the DBXml_Output Stage to the target IncidentReport schema. The result is the IncidentReport Xml document.|
|IISD Output||Returns the IncidentReport Xml document to the Web Service caller.|
The general steps for creating the DataStage job are as follows:
- Create table definitions for all the data that the job will process.
- Design the job by selecting the available stages in the Designer’s palette.
- Compile, run and test.
- Enable the job for Information Services. That means to prepare and make the job available for InfoSphere Information Services Director (IISD) to build and deploy it as a Web service.
Refer to the InfoSphere DataStage and QualityStage Designer Client Guide, SC18-9893-00, for further instruction on developing DataStage job. (See Resources).
The IncRptFedViewParms job uses data retrieved from the IncReport federated view and the myInternalIncRptNoNS.xsd schema. Metadata table definitions need to be created for these two entities.
Create the metadata table definition for the IncReport federated view by using the Import Table Definition wizard. From the menu, select Import -> Table Definition -> Orchestrate Schema Definition or right-click on the Repository panel and select Import Table Definition -> Orchestrate Schema Definition. Figure 18 illustrates these steps:
Figure 18. Create the metadata table definition for the IncReport federated view
When the Orchestrate Schema Definition wizard completes the import, it creates the table definition for the IncReport federated view.
Refer to Chapter 8 "Table Definitions" in InfoSphere DataStage and QualityStage Designer Client Guide, SC18-9893-00, for detail configuration instructions on defining metadata table (see Resources).
Figure 19. The IncReport federated view metadata table definition
You need to create a metadata definition for the myInternalIncRptNoNS.xsd schema as well. To do that, from the menu, select Import -> Table Definition -> XML Table Definition, or right-click on the Repository panel and select Import Table Definition -> XML Table Definition. The Import XML Table Definition wizard starts the XML Meta Data Importer tool which is part of the DataStage XML Pack.
Refer to InfoSphere DataStage Version 8 XML Pack Guide, SC18-9947-00, for instructions on using the XML Meta Data Importer tool (see Resources).
Figure 20. The XML Meta Data Importer
From the XML Meta Data Importer menu bar, select File -> Open -> File to locate the myInternalIncRptNoNS.xsd schema file. After the XML Meta Data Importer has successfully loaded the schema, you will see a display similar to that in Figure 21. The top left panel displays the schema in a tree-like structure. Select the check box next to the data element you want to have included into the table definition. When you click on the check box, you will see the data element display in the Table Definition panel. For each data element that you select, you may also need to made changes to the values of SQL Type, Length and the like.
Figure 21. Select the data fields to be included in the table definition
Figure 22 shows the table definition after that the XML Meta Data Importer finishes importing the XML schema. Note the XPath under the Description column. The XML_Output Stage uses the XPaths to generate the xml document.
Figure 22. Metadata table definition for the legacy data XML schema
To create a DataStage job, drag the Stage from the Palette panel and drop it onto the job canvas, as Figure 23 illustrates:
Figure 23. Select the stage to be included in the DataStage job
The DB2 API stage (incReport) retrieves data from the IncReport federated view. It needs to know the metadata definitions of the data it will be retrieving. Figure 24 shows the properties of the DB2 API stage. You go to the Column tab and select the Load button. That prompts you through the steps of loading the IncReport federated view table definition that you have previously defined. Defining the metadata table definition and loading it into the stage is very typical of what you need to do to configure a DataStage stage.
Refer to InfoSphere DataStage Version 8 Connectivity Guide for DB2 Databases, SC18-9932-00, for detail configuration instructions about the DB2 API stage (see Resources).
Figure 24. incReport Stage column definitions
Figure 25 shows the SQL Select statement used to retrieve the data. The DB2 API stage has a SQL Builder wizard to help you construct the SQL statement. Note the #IncidentNumber# in the WHERE clause. That is a job parameter.
Figure 25. IncReport Stage query properties
You define the job parameter via the Job Properties window as Figure 26 illustrates. Every time the DS job runs, it prompts you for the parameter.
Figure 26. IncRptFedViewParms job parameters
The function of the XML_Output stage is to generate an XML document. In the IncRptFedViewParms job, the XML_Output stage is used for generating the myInternalIncRptNoNS XML document. The XML_Output stage is configured to expect the data elements defined in the myInternalIncRptNONS table definition as input (see Metadata definition for the legacy data XML schema). Figure 27 shows the data columns after the myInternalIncRptNoNS table definition is loaded. Note the XPaths under the Description column.
Figure 27. XML_Output stage’s input page
Figure 28 shows that the XML_Output stage is expected to output one column of data, and the data is the myInternalIncRptNoNS XML document. The “/” under the Description column means the entire xml document.
Figure 28. Xml_Output Stage's output page
Refer to InfoSphere DataStage Version 8 XML Pack Guide, SC18-9947-00, for configuration instructions on XML Output stage (see Resources).
The Transformer stage is used for transforming and mapping the data retrieved from the IncReport stage to the metadata definition defined in the myInternalIncRptNoNS schema. Figure 29 shows the Transformer stage properties:
Figure 29. Transformer stage properties
Refer to Chapter 19 "Transformer Stage" in the InfoSphere DataStage Version 8 Server Job Developer Guide, SC18-9898-00, for detailed instructions on configuring the Transformer Stage (see Resources).
The WebSphere TX Map stage is DataStage’s integration point with the WebSphere Transformation Extender (WTX) product. It enables users of the DataStage product to call WTX and use it to do data mapping and transformation.
You can launch the WTX Type Designer and Map Designer from within the WebSphere TX Map Stage to create and/or edit WTX Type Tree and WTX Map. We created the WTX map (*.mmc file) outside of the DataStage Designer, then copied the entire directory that contained the WTX Type Trees and Maps and pasted it as a subdirectory under the IIS Server WTX map directory (that is, C:\IBM\InformationServer\Server\TXMaps). Figure 31 shows the WTX Map Stage properties:
Figure 30. Map Stage properties
Refer to Chapter 9 "DataStage TX Map Stages" in the InfoSphere DataStage Version 8 Server Job Developer Guide, SC18-9898-00, for detailed instructions on configuring the WTX Map Stage (see Resources).
There is one configuration step within the WebSphere _TX_ Map stage that may not be clear. On both the Input and Output tabs, you have to configure the "Type for this Card" even though the fields are greyed out. Select the drop-down icon next to the Edit button, and fill out the "Configure Link Row and Column Processing for Type" dialog box. Since our data was an XML document, we selected Raw Data.
Figure 31. Input and output cards configuration
The IISD Output Stage enables a DataStage job for information services. Once a IISD Output or Input Stage is added to a DataStage job, the control for running the DataStage job is handled by the Information Services Director instead of the DataStage Director. For the ease of unit testing the majority of the IncidentReport job logic, we deferred adding the IISD Output Stage until the step described in the section "Enable job for Information Services".
To compile the IncidentReport DataStage job:
- Open the DataStage job. The filename for the IncidentReport job is IncRptFedViewParm.
- Select File -> Compile or select the Compile job icon from the Toolbar.
At this point, the IncidentReport job has not yet been information-services enabled. Therefore, it's execution is still under the control of the DataStage Director Client. The job contains the DB2 API stage, the Transformer stage, the XML Output Stage and the WebSphere TX Map stage. To run the IncidentReport job, from the DataStage Designer select Tool -> Run Director. This starts up the DataStage and QualityStage Director Client, as Figure 32 illustrates:
Figure 32. DataStage and QualityStage Director
From the DataStage Director, highlight IncRptFedViewParm job and select the run icon from the Toolbar.
The job prompts you to enter an IncidentNumber. At the completion of the job, it outputs the IncidentReport XML document at the location specified by the WTX Map. To find out the location of the XML document, open the WTX Map and edit the Output Card properties. From the Output Card properties window, you will see an entry for Target File Path which contains the filename and path of the IncidentReport XML document. Figure 33 shows the WTX Map Output card properties:
Figure 33. WTX Map Output card properties
Refer to the InfoSphere DataStage Version 8 Director Client Guide, SC18-9894-00, for detail instructions on operating the DataStage Director Client (see Resources).
At this point the IncidentReport job is ready to be enabled for information services. For the IncidentReport job that means:
- Adding the IISD Output stage and configuring the stage to return to the caller the IncidentReport XML document.
- Modifying the Job Properties to select relevant parameters to enable the job for information services.
For a more in-depth discussion on the IISD Output and Input Stages, refer to the InfoSphere Information Services Director documentation.
The IISD Output Stage takes the IncidentReport XML document generated by the WTX Map stage and returns it to the Web service caller. For every data element to be returned to the caller, the data element must be defined in the IISD Output stage's Input Column table as Figure 34 shows. The IncidentReport job had only one data element which was the IncidentReport XML document. It is defined as one very long character string.
Figure 34. IISD Output Stage
As mentioned above, once a DataStage job has been enabled for information services, the control for its execution shifts from the DataStage Director to the Information Services Director. To test an information-services-enabled DataStage job, you need to use the Information Services Director to publish and deploy the job. Then use a Web service-enabled tool to test the job. This is discussed further in the next section .
There are two parameters from the Job Properties window that must be selected to enable the IncidentReport job for information services. They are:
- Allow multiple instance
- Enable for information services
Figure 35 shows these properties:
Figure 35. IncidentReport job properties