Nowadays, the XML format is a common document standard used for message exchange. It is widely accepted in most industries, such as financial services (SEPA and SWIFT Fin) and healthcare (HL7 v3). However, transforming legacy data to the newly defined XML standard is a big challenge for many companies.
IBM WebSphere Transformation Extender is a very powerful tool for migrating legacy data to XML. WebSphere Transformation Extender separates the data transformation logic and business flow logic, which can save on maintenance costs. WebSphere Transformation Extender can help you easily manage changing industry standards.
In this article, learn about the WebSphere Transformation Extender capabilities by using a simplified mapping example that retrieves legacy data from a database and transforms it to an XML file. With the user-friendly, code-free WebSphere Transformation Extender interface, you can quickly visualize and configure the complex data types and mappings.
Download the source code for this article.
To follow along with the examples in this article, you will need:
- IBM WebSphere Transformation Extender Design Studio Version 8.3 or higher
- IBM WebSphere Transformation Extender with Command Server Version 8.3 or higher
- DB2 Version 8 or higher installed
The scenario uses three tables describing customer records from a legacy
database schema. The tables will be created in a DB2 database named
TESTDB. The tables are: CUSTOMER, PHONE, and ADDRESS. The CUSTOMER table
contains basic customer data, such as first name, last name, gender, date
of birth, and so on. It is assumed that each customer may own more than
one phone number, which will be defined in the PHONE table. The PHONE
table contains a cid field that is the foreign
key referring back to the id field in the
CUSTOMER table. It's assumed that each customer might have more than one
addresses, so it will be defined in the ADDRESS table. The ADDRESS table
also contains a cid field that is the foreign
key referring back to the id field in the
CUSTOMER table. Figure 1 shows the database entity
relationship.
Figure 1. Legacy database entity relationship
Your goal in this scenario is to transform each customer record into one XML file. The output XML file should contain basic information, with multiple phone numbers and addresses if available, for one customer. The XML format has been defined in customer.xsd (see Download). To demonstrate that WebSphere Transformation Extender can handle complex XML schema definitions, customer.xsd includes another schema, datatypes.xsd, which includes another schema named detail.xsd.
In the following sections, you'll create a WebSphere Transformation Extender map that will retrieve customer data from DB2 and transform it to an XML file.
To go through the scenario in this article, you will need to prepare the following environment.
- Create a working space named
mywtxunder C:\temp\ - Download the WTX_XML_Example.zip files, and extract it to C:\temp\mywtx. You should see the following files:
- customer.xsd
- datatypes.xsd
- db_creation.sql
- detail.xsd
- locations.xsd
- Open the DB2 Command Window from Windows Start ->
Program -> IBM DB2, as in Figure
2.
Figure 2. Open DB2 Command Window
- In the DB2 Command Window, change to the c:\temp\mywtx directory.
- Create a database named TESTDB using the
db2 create db testdbcommand.Wait a short while, and you should see that the database was created successfully.
- Import
create table, and insert data with thedb2 -tvf db_creation.sqlSQL command.Make sure all the SQL commands are executed successfully.
You should now have three tables: CUSTOMER, ADDRESS, and PHONE. You can validate the data are inserted correctly by selecting from each table. If there are four records inserted into the customer table, six records in the address table, and eight records in the phone table, your database environment is ready.
Create the WebSphere Transformation Extender map
The first step is to create a WebSphere Transformation Extender type tree for the legacy database. WebSphere Transformation Extender provides the user friendly Database Type Tree Designer to help you quickly construct a WebSphere Transformation Extender type tree from the legacy database.
- Open the Database Interface Designer (DID) from the Windows Start menu
by selecting IBM WebSphere Transformation Extender 8.3 ->
Design Studio -> Database Interface Designer, as
in Figure 3.
Figure 3. WebSphere Transformation Extender Database Type Tree Designer
- In the Startup window, select Create a new database/query file
and click OK.
The application will then start a new database file.
- In the DID application, click File -> Save As and save the file to C:\temp\mywtx\db2.mdq.
- In the left pane of the application, under db2, right click on
Database and select New to open a Database
Definition window, as in Figure 4.
Figure 4. Open Database Definition
- In the Database Definition window, fill in the following details, as
shown in Figure 5, then click OK.
- Database Name: DB2Source (this could be any name)
- Type: DB2
- Platform: Microsoft Windows
- Database Interface Designer: TESTDB (this name must match the DB name)
- Runtime: TESTDB (this name must match the DB name)
Fill in the security information if necessary (database user id and password).
Figure 5. Database Definition
- You need to create four new queries in the DID application. To create
one new query, under Database/Query Files -> db2 -> Database
-> DB2Source, right click on Queries. Select New…
from the drop-down menu. Input the Query Name and Query command, as
shown in the table below. Click OK.
Repeat this step to create the other three queries.
Query name Query command Description SelectAll SELECT * FROM CUSTOMERTo select all customer information from the CUSTOMER table. GetCustomer SELECT * FROM CUSTOMER WHERE ID='#CID#'To select one customer record from the CUSTOMER table. GetPhone SELECT TYPE, NUMBER FROM PHONE WHERE CID ='#CID#'To select multiple phone data of a specific customer from the PHONE table. GetAddress SELECT TYPE, STREET, CITY, COUNTRY FROM ADDRESS WHERE CID ='#CID#'To select multiple address details of a specific customer from the ADDRESS table.
#CID#, a variable recognized by the DID application, lets you input the value for the variable at execution time.You should now have four queries available under Database/Query Files -> db2 -> Database -> DB2Source -> Queries.
- Generate three type trees for the newly created GetCustomer,
GetAddress, and GetPhone queries.
Starting with GetCustomer, under Database/Query Files -> db2 -> Database -> DB2Source -> Queries, right click on GetCustomer and select Generate Tree…, as in Figure 6.
Figure 6. Generate Tree
Make sure the Type Tree name is C:\temp\mywtx\GetCustomer.mtt, as in Figure 7, and click OK to generate the type tree.
Figure 7. Generate GetCustomer type tree from query
You should see a message that the command file completed successfully. Close the Generate type tree window.
- Repeat Step 7 to generate two more type trees from the GetAddress and
GetPhone queries.
When completed, you should have three type trees under the c:\temp\mywtx folder: GetCustomer.mtt, GetAddress.mtt, and GetPhone.mtt.
- Save the file using File -> Save. Close the DID.
Create a new WebSphere Transformation Extender Project
To create a new WebSphere Transformation Extender project:
- Select IBM WebSphere Transformation Extender 8.3 ->
Design Studio -> Design Studio, as shown in Figure 8.
Figure 8. Open WebSphere Transformation Extender Design Studio
- In the Workspace Launcher window, enter
c:\temp\mywtx\workspacefor the workspace. - Create a new WebSphere Transformation Extender project by selecting
File -> New -> Extender Project, as in
Figure 9.
Figure 9. Create a new WebSphere Transformation Extender project
- Enter
DB2XMLas the project name and click Use default location. Click Finish.A new project will then be created in the Extender Navigator.
Import input DB type tree and output XML schema
- In the Extender Navigator, right click on Type Trees and select Import -> File System.
- In the Import wizard, as shown in Figure 10, use
C:\temp\mywtxfor the From directory. In the right pane, select:- customer.xsd
- datatypes.xsd
- db2.mdq
- detail.xsd
- GetAddress.mtt
- GetCustomer.mtt
- GetPhone.mtt
- location.xsd
Figure 10. Configure Import wizard for DB type tree
- Click Finish to import the database type tree into the
WebSphere Transformation Extender project. As shown in Figure 11, you should see:
- db2.mdq under Database Files
- customer.xsd, datatypes.xsd, detail.xsd, and location.xsd under Schema Files
- GetAddress.mtt, GetCustomer.mtt, and GetPhone.mtt under Type Trees
Figure 11. Imported files available from Extender Navigator
Because WebSphere Transformation Extender supports XSD schema directly, you do not need to transform to type tree.
Configure a WebSphere Transformation Extender map
Thus far you've prepared type trees for legacy data and imported XML schema for standard output. In this section, you will configure a WebSphere Transformation Extender map to transform the data from the legacy database to XML output.
- In the DB2XML project of WebSphere Transformation Extender Design
Studio -> Extender Navigator, right click on DB2XML and
select New -> Map Source, as in Figure 12.
Figure 12. Create a new map
- In the Map Source wizard, provide the File name
DB2XMLMapand click Finish to generate an empty map source, as in Figure 13.
Figure 13. Configure Map Source wizard
- In this mapping, you need to create two functional maps and one executable map. One of the functional maps is used to perform mapping for address detail, and the other is used to perform mapping for phone detail. Let's start with creating a functional map.
- To create the functional map F_ADDRESS:
- In the outline view, right click on DB2XMLMap and
select New, as in Figure 14. In
the pop-up window, provide the New Map name
F_ADDRESS.
Figure 14. Configure a new executable map
- In the Outline view, under F_ADDRESS map, right click on
Input Cards and select New. Input the
following information for the Input Card, as in Figure 15. Leave the defaults in the
other fields.
- CardName: AddressIn
- TypeTree: GetAddress.mtt
- Type: Row GetAddress Data
In the Select Type window, click on Data -> GetAddress -> Row, then click OK. The Type will show in reverse order, as in Figure 15. The same applies to the Type selection in this article.
Figure 15. Input Card configured for F_ADDRESS map
- Right click on Output Cards and select New.
Enter the following information for the Output card. Leave the
defaults in the other fields, as shown in Figure 16.
- CardName: AddressOut
- TypeTree: customer.xsd
- Type: address sequence customer global XSD
Figure 16. Output Card configured for F_ADDRESS map
- After you've created input and output cards, you can see the
two cards available on the map view on the right side of the
window. Drag and drop the TYPE, STREET, CITY, and COUNTRY
fields from AddressIn to the AddressOut card, as shown in Figure 17.
Figure 17. Mapping in F_ADDRESS
- In the outline view, right click on DB2XMLMap and
select New, as in Figure 14. In
the pop-up window, provide the New Map name
- Create the F_PHONE functional map similarly to the previous steps.
- On the left side of the Outline view, right click on
DB2XMLMap and select New. Provide the new
map name
F_PHONE. - In the Outline view, under the F_PHONE map, right click on
Input Cards and select New. Enter the
following information for the Input card. Leave the defaults
in the other fields.
- CardName: PhoneIn
- TypeTree: GetPhone.mtt
- Type: Row GetPhone Data
- Right click on Output Cards and select New.
Enter the following information for the output card. Leave the
defaults in the other fields.
- CardName: PhoneOut
- TypeTree: customer.xsd
- Type: phone_number sequence customer global XSD
- After creating the input and output cards, you should see them
in the map view on the right of the window. Drag and drop TYPE
and NUMBER from PhoneIn to the PhoneOut card, as in Figure 18.
Figure 18. Mapping in F_PHONE
- On the left side of the Outline view, right click on
DB2XMLMap and select New. Provide the new
map name
- Create the DB2XML executable map. (Unlike the functional map, for the
executable map you need to define the input/output source type, such
as file, db2, mq, and so on.) The executable map can be executed to
process input data.
- On the left side of the Outline view, right click on
DB2XMLMap and select New. Name the new map
DB2XML. - Create three input cards for the executable map to get data
from the CUSTOMER, PHONE, and ADDRESS tables. In the Outline
view, under DB2XML map, right click on Input Cards and
select New to create the first input card to get
customer data. Enter the following information for the input
card. Leave the defaults in the other fields.
- CardName: GetCustomer
- TypeTree: GetCustomer.mtt
- Type: DBSelect GetCustomer Data
- Source: Database
- Command: -T (Used to print the DB2 trace for troubleshooting if necessary)
- DatabaseQueryFile
- FilePath: db2.mdq
- Database: DB2Source
- Query: GetCustomer
- In the same way, add the second input card to get address
data. Enter the following information for the input card.
Leave the defaults in the other fields.
- CardName: GetAddress
- TypeTree: GetAddress.mtt
- Type: DBSelect GetAddress Data
- Source: Database
- Command: -T
- DatabaseQueryFile
- FilePath: db2.mdq
- Database: DB2Source
- Query: GetAddress
- Similarly, add the third input card to get phone data. Enter
the following information for the input card. Leave the
defaults in the other fields.
- CardName: GetPhone
- TypeTree: GetPhone.mtt
- Type: DBSelect GetPhone Data
- Source: Database
- Command: -T
- DatabaseQueryFile
- FilePath: db2.mdq
- Database: DB2Source
- Query: GetPhone
- Right click on Output Cards and select New.
Enter the following information for the output card. Leave the
defaults in the other fields.
- CardName: XMLOut
- TypeTree: customer.xsd
- Type: XSD
- Target: File
- FilePath: customer_out.xml
- After you've created the input and output card, drag and drop
the fields from the input card to the output card. Figure 19 shows the mapping for
GetCustomer.
Figure 19. GetCustomer input card to output card
Figure 20 shows the mapping for GetPhone and GetAddress.
Figure 20. GetPhone and GetAddress input card to output card
The table below shows the detailed mapping information.
Output Card field Value XMLOut Misc Pcdata =NONE Prolog Version ="1.0" Encoding ="UTF-8" Standalone ="yes" Global Customer Userid =ID Column:Row:GetCustomer Sequence first_name =FIRST_NAME Column:Row:GetCustomer middle_name =MIDDLE_NAME Column:Row:GetCustomer last_name =LAST_NAME Column:Row:GetCustomer id =SOCIAL_ID Column:Row:GetCustomer gender =GENDER Column:Row:GetCustomer birthdate*
* This field needs to apply DATETOTEXT function=DATETOTEXT(BIRTHDATE Column:Row:GetCustomer) occupation =OCCUPATION Column:Row:GetCustomer phone_number (1:s) =F_PHONE(Row:GetPhoned) Address (1:s) =F_ADDRESS(Row:GetAddress)
For the phone_number and address fields, type in the Rule panel as shown above. F_PHONE and F_ADDRESS are the names of function maps that process rows of data retrieved by the GetPhone and GetAddress queries.
- Save the map.
- On the left side of the Outline view, right click on
DB2XMLMap and select New. Name the new map
- To build the map, select the DB2XML map. From the File menu select
Map -> Build for Specific Platform ->
Microsoft Windows, as in Figure 21.
Figure 21. Compile the map
Run the WebSphere Transformation Extender Map
To execute the map using Command Server:
- Open a DOS window from Windows Start -> Run. Enter
cmd. - In the DOS window, change directories to where WebSphere Transformation Extender is installed. (For WebSphere Transformation Extender 8.3, it should be installed under C:\IBM\WebSphere Transformation Extender 8.3.)
- Execute the DB2XML map that you built in the previous section using
the following command.
dstx C:\temp\mywtx\workspace\DB2XML\DB2XML.mmc -ID1 '-VAR CID=000001' -ID2 '-VAR CID=000001' -ID3 '-VAR CID=000001'-AE -TIO
Because the example uses the variable
CIDin the DID, you can assign the value to the variable from the Command Server (above). If you encounter errors, the-AEargument is for printing an audit log and the-TIOargument is for printing a trace log of both input and output cards.You should see a window similar to Figure 22 that shows the map completed successfully.
Figure 22. Map execution result
The output file named customer_out.xml should be under C:\temp\mywtx\workspace\DB2XML. Open the XML file, which should contain the customer data shown below.
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?> - <customer userid="000001"> <first_name>Bob</first_name> <middle_name>JK</middle_name> <last_name>Smith</last_name> <id>AAAAAAAAAA</id> <gender>M</gender> <birthdate>1977-01-01</birthdate> <occupation>Engineer</occupation> <phone_number type="office">111-1111</phone_number> <phone_number type="mobile">0911-123456</phone_number> - <address type="home"> <street>1 Park Street</street> <city>New York</city> <country>U.S.</country> </address> - <address type="office"> <street>1 Lakewood Street</street> <city>New York</city> <country>U.S.</country> </address> </customer>
- If you encounter problems and don't get the output file shown above,
try the following troubleshooting steps.
- Turn on the map's audit log and trace from Map Settings to check WebSphere Transformation Extender error messages. Open the Map Setting configuration by right clicking on the DB2XML map and selecting Map Settings. After configuring the map settings, recompile the map to enable it.
- Look for the database trace named *.dbl under the C:\temp\mywtx\workspace\DB2XML directory. It will show the database activities, SQL queries, and error messages (if any).
You've learned how to convert one customer record from a legacy database to an XML file. In this section, convert all the customer records from a legacy database to XML files. You need to create one additional type tree and two maps.
- In WebSphere Transformation Extender Design Studio create a new type
tree with File -> New -> Type Tree. In the
New Type Tree window, select DB2XML as the parent folder, give
the type tree name
OutputString, and click Finish.On the OutputString type tree, right click on Root and select Add to add a new type named Output. Add another new type named Outputs. Configure the Properties of these two items with the following information. Leave the default values in the other fields.
Output:
- Name: Output
- Class: Item
- Item Subclass: Text
Outputs
- Name: Outputs
- Class: Group
- Double click on the Outputs group to open the component editor.
Drag and drop the Output item to the component column, as shown in Figure 23.
Figure 23. Add item in Outputs group Component Editor
Right click on Output in the Component column and select Set Range, as shown in Figure 24. In the pop-up window, set Min to
1and Max tos. Save the component editor.Go back to the OutputString type tree, analyze the tree from File menu -> Tree -> Analyze -> Structure and Logic. There should be no error in the analyzed result. If you see a warning that shows:
.L201-Different data objects of COMPONENT 1 may not be distinguishable in Type 'Outputs Root' (warning)
You may ignore it. Save and close the type tree.
Figure 24. Select Set Range for Output item
- From the Outline view, right click on DB2XMLMap and select New. Name the new map F_RUN.
- In the Outline view, under the F_RUN map, right click on Input
Cards and select New. Enter the following information
for the input card. Leave the defaults in the other fields.
- CardName: CustomerRow
- TypeTree: GetCustomer.mtt
- Type: Row GetCustomer Data
- Right click on Output Cards and select New. Enter the
following information for the output card. Leave the defaults in the
other fields.
- CardName: Output
- TypeTree: OutputString.mtt
- Type: Output Root
- After you've created the input and output card, you can see the cards
in the Map view on the right. Click on the Output card and
enter the following information in the Output item, as shown in Figure 25.
=VALID(RUN("DB2XML.mmc", "-ID1 '-VAR CID="+ID Column:CustomerRow+"' -ID2 ' -VAR CID="+ID Column:CustomerRow+"' -ID3 '-VAR CID="+ID Column:CustomerRow+"' -OF1 "+ID Column:CustomerRow +".xml -TIO"), FAIL("My run failed!"))
Figure 25. Mapping in F_RUN function
This rule calls the Run function to execute the map, DB2XML.mmc, which you just built, and sets options for the CID variable. The VALID function will then check whether the RUN is successful. If the RUN fails, the VALID function calls the FAIL function to terminate F_RUN map.
Create the RunMap executable map
To create an executable map named RunMap:
- On the left side of the Outline view, right click on DB2XMLMap
and select New. Name the new map
RunMap. - Create an input card to get all the customer records from the
database. In the Outline view, under RunMap map, right click on
Input Cards and select New to create the input card.
Enter the following information for the input card. Leave the defaults
in the other fields.
- CardName: AllCustomer
- TypeTree: GetCustomer.mtt
- Type: DBSelect GetCustomer Data
- Source: Database
- Command: -T (To print out the DB2 trace for troubleshooting if necessary)
- DatabaseQueryFile
- FilePath: db2.mdq
- Database: DB2Source
- Query: SelectAll
- Right click on Output Cards and select New. Enter the
following information for the output card. Leave the defaults in the
other fields.
- CardName: FileOut
- TypeTree: OutputString.mtt
- Type: Outputs Root
- Target: File
- FilePath: Test.out
- After you've created the Input and Output cards, in the mapping editor
enter the following data in the Output map.
Output Card field Value FileOut Output (1:s) =F_RUN(Row:AllCustomer)
You can also see the mapping in Figure 26.
Figure 26. Mapping in RunMap map
- Save the map.
- Right click on RunMap and select Build to build the map.
You should not see any errors after building the map.
- Right click on RunMap and select Run to run the map. You
should see the Command Server window showing "Map completed
successfully," as in Figure 27.
Figure 27. RunMap is executed successfully
Open Windows Explorer and check the C:\temp\mywtx\workspace\DB2XML directory, where you should see four newly generated customer records in xml format: 000001.xml, 000002.xml, 000003.xml, and 000004.xml. Instead of getting only one detail about a customer, you can now get all the customer details from the legacy database in XML format.
The scenario in this article walked you through the process for creating a WebSphere Transformation Extender map to retrieve legacy data from a database and transform it to an XML file. Using WebSphere Transformation Extender interface you quickly configured the complex data types and mappings.
| Description | Name | Size | Download method |
|---|---|---|---|
| Sample code for this article | WTX_XML_Example.zip | 2.35KB | HTTP |
Information about download methods
Learn
- WebSphere
Transformation Extender: Get product descriptions, news, training
information, support information, and more. Specifically:
- Documentation: Use the Information Center for all WebSphere Transformation Extender documentation, with conceptual, task, and reference information on installation, configuration, and usage.
- Library: Get all the WebSphere Transformation Extender information, including books, and release notes for recent versions.
- Requirements: Learn about the hardware and software requirements.
- Support: Start here to search a database of problems and solutions, plus downloads, fixes, and problem tracking.
- developerWorks
WebSphere zone: Get technical information and resources, product
downloads, how-to information, support resources, and a free technical
library of more than 2000 technical articles, tutorials, best practices,
IBM Redbooks, and online product manuals.
- developerWorks WebSphere application integration zone: Explore
how-to articles, downloads, tutorials, education, and product information,
to help you build WebSphere application connectivity and business
integration solutions.
- developerWorks WebSphere Application Server zone: Get how-to
articles, downloads, tutorials, education, and product information to help
you design and build WebSphere SOA and Web services solutions.
- WebSphere downloads: Access WebSphere downloads, including
product trials, emerging technologies, updates, fixes, utilities and
drivers.
- WebSphere on-demand demos: With self-running demos, learn how
WebSphere products and technologies can help you respond to the rapidly
changing and increasingly complex business environment.
- developerWorks
WebSphere weekly newsletter: Connect to the latest and greatest
developer resources. Get the latest articles and information on topics
that interest you.
- WebSphere-related books from IBM Press: Peruse the latest
literature and link to convenient online ordering.
- WebSphere-related events: Learn about conferences, trade shows,
Webcasts, and other events around the world of interest to WebSphere
developers.
- IBM Education Assistant: See the multimedia educational modules
that will help you better understand IBM software products and use them
more effectively to meet your business requirements.
Get products and technologies
- Build your next
development project with IBM trial
software, available for download directly from
developerWorks.
Discuss
- WebSphere Transformation Extender forum: Share answers and
solutions for transformations.
- WebSphere forums: Address technology- and product-related issues.
Get answers to your technical questions and share your expertise with
other WebSphere users.
- Check out the developerWorks blogs and get involved in the developerWorks
community.

Elisa Su is a staff software engineer at China Software Development Lab, IBM Taiwan on the IBM IMS GVT team. She is the testing leader for the Globalization Verification Testing against Message and Data Services component of IMS. Prior to that, she was the development leader for the IBM worldwide technical support site. Her areas of expertise include J2SE/J2EE programming, Web application development, content management, and project management.

Jervis Lee is a staff software engineer at the IBM Taiwan Software Development Lab. He is the technical leader of WebSphere Transformation Extender Performance Testing team. His areas of expertise include performance testing, J2SE/J2EE/J2ME programming, web application development, and information security solutions.

Owen Chang is an advisory I/T specialist in the software group, IBM Taiwan. He is the technical support specialist for the WebSphere team. Prior to that, Owen was the service project manager for the IBM Rational software service. His areas of expertise include connectivity solutions, QA management, and project management.




