Specifying multiple inputs for a binary transformation using a Websphere Transformation Extender map with WebSphere DataPower

When using Websphere Transformation Extender (WebSphere TX) maps on DataPower, the typical transformation task is to transform a file from one format to another without any augmentation or data validation steps. When a second input is necessary, the DataPower Policy Rule requires a number of changes to enable multiple inputs or outputs. This article shows you how to configure DataPower to execute the WebSphere TX Map with multiple inputs.

Share:

Charlie Sumner (csumner@us.ibm.com), WebSphere Consultant, IBM

Photo of Charlie SumnerCharlie Sumner is a Websphere pre-sales consultant based in Boca Raton, Florida. Charlie began his career with IBM on mainframe systems but quickly switched to personal and mid-sized systems and spent the majority of his career as a developer in Speech Recognition systems. He currently focuses his consulting work on WebSphere Transformation Extender and its integration with other WebSphere products.



28 April 2010

Introduction

This article describes the setup required to use multiple inputs to an IBM® WebSphere® TX Map (hereafter called WebSphere TX) on IBM WebSphere DataPower® (hereafter called DataPower). For simplicity, the cross-reference file is stored and retrieved from memory on DataPower. The secondary input can be stored anywhere accessible via a URL and can even be a Datapower system variable, which is then copied to a DataPower context variable.

To benefit from this article, you should have at least an introductory knowledge of WebSphere TX. A step-by-step procedure is described, but details of using the WebSphere TX Design Studio are only touched on briefly. The procedure has been tested on the DataPower Integration Appliance XI50, and will probably work on the DataPower B2B Appliance XB60, but it has not been tested on the XB60. At the bottom of this article, you can download an exported project interchange file that contains all of the artifacts described in the article.

The scenario involves a fixed-column-length flat file as the primary input, and an XML file as the output. A secondary input file containing a list of US state codes and names is used as a cross-reference file. The primary input file was originally described by a COBOL copybook and the secondary input file has no machine-readable description. The output file has an XML schema. The task is to read each record of the input file, look up the US state code in the cross-reference file, and retrieve the US state name. Other tasks are included in this transformation, but they are not related to multiple input streams. Here are examples of the input and output files:

Primary input file
Jones Charlie 123 Main St. Boca RatonNY334311234H5615551212W5618622000M5618629999
Secondary input file
FLFLORIDA
NJNEW JERSEY
NYNEW YORK
Output file
<Address_Info>
<Last_Name>Jones</Last_Name>
<First_Name>Charlie</First_Name>
<Address>123 Main St.</Address>
<City>Boca Raton</City>
<State>NEW YORK</State>	
<ZIP_Plus_4>33431-1234</ZIP_Plus_4>
<Phones type="H">
<Phone_Number>5615551212</Phone_Number></Phones>
<Phones type="W">
<Phone_Number>5618622000</Phone_Number></Phones>
<Phones type="M">
<Phone_Number>5618629999</Phone_Number></Phones>
</Address_Info>

The secondary input cross-reference file is uploaded and stored in the local DataPower store. The primary file is supplied to DataPower as an input HTTP request. The WebSphere TX map is inserted into a DataPower policy binary transformation rule. The policy will need to be updated to:

  1. Change the setting Look for named inputs and output.
  2. Create two named inputs the same name as the WebSphere TX input cards.
  3. Create a named output.
  4. Fetch the cross-reference file and store it in a context variable.

Prerequisites

  • Login access to a DataPower device with your own domain and a WebSphere TX domain installed
  • WebSphere TX Design Studio installed -- a simple one-to-one map and type trees
  • The CURL tool: Download a free trial version.

Create the multi-input WebSphere TX map

  1. Start with a simple WebSphere TX map that takes a flat file and transforms it into an XML document. In the Design Studio, add a second input card to use as a cross-reference file:
    Two input (From) cards
    Two input (From) cards
  2. In order to display the two input cards, you may have to change an Eclipse default setting to display more than one input or output card. Under Windows, select Preferences => WebSphere Transformation Extender => Map. Change Maximum number of From cards to 3, and Maximum number of To cards to 3:
    Eclipse WebSphere TX map settings
    Eclipse WebSphere TX map settings
  3. As stated above, the cross-reference file is used to identify a US state name using a US state code supplied with the input. The output state name is created using a map rule and the lookup function, which has two parameters: the first one to specify the series element to return, and the second one the test to perform. In this case, the series elements are the US state names, and the test is to perform is a match between the input US state code and the list of US state codes provided by the cross-reference file. The LOOKUP function is coded like this:
    =LOOKUP(StateName Fields:StateRecord:States,
        StateCode Fields:StateRecord:States=STATE Field:Cardin)

    As you can see, the name of the primary input card is Cardin and the name of the secondary input or cross-reference file is States. This information is important for later steps.

  4. After you have completed the map, build and test it on DataPower using the Design Studio Run function. Under Map settings, change the Map Runtime to WebSphere DataPower:
    Map settings changed to DataPower
    Map settings changed to DataPower
  5. Next, configure Eclipse to send the artifacts to DataPower for execution and return of the results. In the Preferences window, select WebSphere Transformation Extender => Map => Datapower. Configure your DataPower device's host name and port number:
    Eclipse preferences
    Eclipse preferences
  6. At this point, you can run the WebSphere TX map on DataPower for testing. Although you have specified two inputs, you can still test the map to make sure you are getting the correct results.

Now comes the interesting part -- deploying your multi-input map to DataPower and configuring it to accept multiple inputs.

Configure DataPower to run the WebSphere TX map

The following steps require a sign-on to DataPower and preferably your own domain. The DataPower device can be shared among many users and applications through the use of a domain separation scheme. The first step in configuring the DataPower device is to create an XML firewall, which is basically the simplest DataPower object to accept a client request and produce a response. To configure and test the DataPower device to run the multiple input map that you just created:

  1. Create the XML firewall.
  2. Upload the cross-reference file and WebSphere TX map.
  3. Configure the policy:
    1. Fetch the cross-reference file.
    2. Create the binary transform.
  4. Configure the binary transform to use named inputs and outputs.
  5. Configure the named inputs and outputs.
  6. Test the configuration using the CURL tool.

Create the XML firewall

The XML Firewall Creation Wizard makes it much easier to create an XML firewall:

  1. Log in to the the DataPower device using your credentials and switch to your domain.
  2. From the DataPower Control Panel, click on the XML Firewall icon:
    XML firewall icon
  3. You may see a list of existing firewalls if there are some already configured, but click Add wizard at the bottom of the list.
  4. Click Non-XML at the bottom of the list and then click Next.
  5. Enter a name for your new firewall -- no spaces or special characters. Then click Next.
    XML firewall name dialog
    XML firewall name dialog
  6. Change the Firewall type to Loopback proxy and click Next.
  7. There are no changes to the Front end (client) information page, but make note of the Device Port, which you will use to access the device via your firewall. Click Next.
  8. Here is the confirmation page: .
    XML firewall confirmation page
    XML firewall confirmation page
  9. Click Commit.
  10. On the next page, click Done. The XML firewall has been created successfully!

Upload the cross-reference file and WebSphere TX map

Two files need to be uploaded from your machine to the DataPower device: the cross-reference file, and the WebSphere TX generated map. Use the DataPower file manager to upload these two files:

  1. From the DataPower control panel, click on the File Management icon:
    File management icon
  2. On the File management dialog, click Actions… next to the Local folder and select Upload files:
    File management display
    File management display
  3. On the next dialog, click Browse and navigate to your WebSphere TX Design Studio workspace, then to your project. Select your wtxmapname.dpa file and click Open.
  4. Click Add to add the map file to the upload queue and open up another upload line to add a file.
  5. Click Browse and navigate to your secondary input file or cross-reference file. Click Open, and then click Add.
  6. Click Upload. You should get a confirmation screen saying that both files have uploaded successfully. Click Continue.

Configure the policy

In this step, you will configure the steps that DataPower will use to retrieve the cross-reference file, execute the binary transform, and then return the output to the requestor:

  1. From the DataPower Control Panel, click XML Firewall.
  2. Click on the XML Firewall that you just created.
  3. In the Configure Firewall screen, click on the Ellipses button on the Firewall Policy:
    Configure the firewall policy
    Configure the firewall policy
  4. Two actions are already configured for you: Match and Output. Leave them as-is.
  5. Drag the Advanced icon to the line between the Match and Output actions:
    Drag Advanced action to line
    Drag Advanced action to line
  6. Double-click the Advanced action that you just dragged to the line. The Configure action dialog is displayed.

Read the secondary file and assign the contents of the file to a context variable:

  1. Scroll down to Fetch, click the Fetch radio button, and click Next. The Fetch configuration screen is displayed.
  2. At the top of the screen, click Advanced, which is necessary when the cross reference file is not XML.
  3. In the upper Source drop-down, select local:///. In the lower Source drop-down, select the name of the cross-reference file that you uploaded.
  4. In the Output Type drop down, select Binary and click Done.
  5. In the Output field, Enter var://context/PoC/myXREF and click Done.

Build the binary transform action, which is the WebSphere TX map that you created above:

  1. Drag the Transform action in between the Fetch and Output actions:
    Drag Transform action to line
    rag Transform action to line
  2. Double-click the Transform action on the line. In the Configure Transform Action screen, click Use XSLT specified in this action on a non-XML message, which turns the page into the Configure Transform Binary action.
  3. On the lower WebSphere TX map file drop-down, select the map file that you uploaded earlier. Make sure WebSphere TX map mode is set to DPA, and then click Done.
  4. Click Apply Policy and then click Close Window:
    Apply the policy and then close window
    Apply the policy and then close window
  5. Click Apply on the Configure XML Firewall screen.

Configure binary transform to use named input and outputs

When you configured the binary transform above, we ignored the inputs and outputs. The settings in this dialog are used only when there are single inputs or outputs. This scenario has multiple inputs, so the inputs and outputs on the binary transform dialogs are ignored and must be explicitly named, as shown below:

  1. On the Navigator on the left side of the screen, click Objects, scroll down to the XML Processing section, and click Processing Action:
    Processing Action navigation item
    Processing Action navigation item
  2. In the list of Processing Actions, look for the name of your firewall followed by _request_xformbin_0, then click on it.
  3. In the Configure Processing action, change the Locate Named Inputs and Outputs from default to Explicit:
    Locate Named Inputs and Outputs setting
    Locate Named Inputs and Outputs setting
  4. Near the top of the screen, click Named Inputs and then click Add:
    Entering named inputs
    Entering named inputs
  5. A window opens. In the Name field, enter the name of the Primary Input card Cardin. In the Context field type, enter INPUT (in upper case). Then click Apply.
  6. Click Add again, and in the Name field, enter the name of the secondary Input card States.
  7. In the Context field, enter var://context/PoC/myXREF and click Apply.
  8. Near the top of the screen, click Named Outputs and then click Add.
  9. A window opens. In the Name field, enter OutputCard or the name of your output card. In the Context field, enter OUTPUT, which is the DataPower standard context output variable.
  10. Click Apply and then click Apply again on the next screen.
  11. In the upper right corner, click Save Config to save the configuration changes to your domain.

The configuration of the DataPower device is complete. The next step is to test the configuration:

Test and troubleshoot the completed configuration

The CURL tool is a free download that lets you send an HTTP request in the form of a file to any URL and then display the response. To send the primary input file to the firewall that you just configured using CURL:

  1. Open a command prompt window and change directory (CD) to to your WebSphere TX Design Studio workspace, then navigate to your project.
  2. Type the CURL command: CURL --data-binary @inputfilename DataPowerhostname:portnumber and press Enter. For example: CURL --data-binary @TestData.txt myDPMachine.ibm.com:2055

    Note the two dashes before the word data and one after, and the @ sign before the filename and the space after it.

  3. If you configured your Firewall correctly, you should see the desired output returned to you from the CURL command. Note the State element retrieved from the cross-reference file.

Since there were more than 40 steps to create this scenario (not counting the WebSphere TX pieces), the first step in troubleshooting is to review the steps and make sure that you didn't miss a step or make a typo on one of the named inputs or outputs. Here are a two other common problems:

FaultString = Malformed Content
The most likely cause of this problem is that the Fetch action was set to an output type of Default or XML instead of Binary.
FaultString = Internal Error
The most likely cause of this problem is that the named inputs or outputs do not match the WebSphere TX input or output cards. Make sure the case is correct and make sure you have an named output (even though there is only one).

Acknowledgements

The author would like to thank IBM IT Specialist Dave Mulley for his help in reviewing the article and testing the procedures in it.

Instructions for using the download file

The exported project interchange file MultipleInputsonDataPower.zip contains all of the artifacts described in the article. To download it, click on HTTP below. To use the artifacts in it:

  1. Save the Project Interchange file MultipleInputsonDataPower.zip to your hard disk.
  2. Open the WebSphere TX Design Studio.
  3. Select File => import => Other => Project Interchange.
  4. Select the project folder ICArtifacts to import and click Finish. The Extender project will appear in the Extender Navigator.

Download

DescriptionNameSize
Code sampleMultipleInputsonDataPower.zip9 KB

Resources

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere
ArticleID=485750
ArticleTitle=Specifying multiple inputs for a binary transformation using a Websphere Transformation Extender map with WebSphere DataPower
publish-date=04282010