Process JSON data with WebSphere Transformation Extender

Transform and validate JSON data

IBM® WebSphere® Transformation Extender validates and transforms complex data. With this article, learn how to use a generic JSON type tree to write type trees that parse and validate specific JavaScript Object Notation (JSON) data.

Share:

Lefty A. Rivera-Correa (lefty@us.ibm.com), Advisory Software Engineer, IBM

Lefty Rivera-Correa photoLefty A. Rivera-Correa is an Advisory Software Engineer and member of the IBM WebSphere Transformation Extender team. Lefty works on Design Studio and many of the other Transformation Extender integrations with IBM products. Lefty has been with IBM since 2001.



08 November 2012

Overview

This article includes an overview of WebSphere Transformation Extender and an exercise. The exercise demonstrates how to use a generic JSON type tree in your map to validate any JSON data with WebSphere Transformation Extender. The generic type tree structure complies with the JSON data definition as defined by RFC 4627.

The first part of the article provides the prerequisite steps for completing the exercise and describes the structure and function of the generic JSON type tree. The second part describes how to create a type tree for a purchase order JSON message. To follow the steps in the exercise, you need some experience with WebSphere Transformation Extender.

What is WebSphere Transformation Extender?

WebSphere Transformation Extender is IBM's universal transformation engine. WebSphere Transformation Extender Design Studio is a graphical user interface that you use to create type trees and maps. For more information about type trees and maps, see the WebSphere Transformation Extender information center documentation in the Resources section in this article.

After you define the WebSphere Transformation Extender Design Studio maps, you can run them by using:

  • Batch mode
  • APIs for various languages
  • Web service call
  • z/OS® operating system

Additionally, WebSphere Transformation Extender is used with other IBM products, including:

  • WebSphere Message Broker
  • WebSphere Enterprise Service Bus
  • WebSphere Process Server
  • WebSphere DataPower®
  • IBM Business Process Manager
  • IBM Sterling B2B Integrator®

Preparing for the JSON example

To successfully use this example, you need to complete the following prerequisite steps:

  1. Install WebSphere Transformation Extender Design Studio, version 8.4.0.2 or later. For a link to the WebSphere software downloads Web site, see the Resources section in this article.
  2. Download the WTXJSONSetup.zip file from the Downloads section in this article. The .zip file contains the following directory structure:
    • copybook
      • PurchaseOrder.cpy
      • PurchaseOrder.mtt
    • json
      • PurchaseOrder.json
      • PurchaseOrder.mtt
    • wtx
      • JSON.mtt
      • PurchaseOrder.mms
    Figure 1. Contents of the zip file
    Contents of the WTXJSONSetup zip.
  3. In the Design Studio, use the Import wizard to import the file into your workspace project as a Project Interchange file:
    1. Select File > Import.
    2. Open the Other folder, select Project Interchange > Next.
    3. Follow the Import wizard instructions.

Understanding the generic JSON type tree

To better understand this example, you must understand the distinct branches of the generic JSON type tree. The generic JSON.mtt type tree is divided into: generic branch, input branch, message type, and output branch.

Figure 2. Generic JSON type tree
Generic JSON type tree.

Generic branch

The generic branch contains the boolean and null values, which are the same for both input and output processing.

Figure 3. Generic branch
Generic branch contains: Boolean and Null.

Input branch

The input branch is used to validate JSON data and is divided into the following two parts:

  • Internals – contains the internal types that describe the JSON data format.
  • Types – contains the main JSON data types, including number, string, value, member (name-value pair), and holders for array and object definitions.
Figure 4. Input branch
Input branch contains: Internals and Types.

Message type

The message JSON type is the type that the map uses for generic JSON data validation.

Figure 5. Message type
Message type is Message

Output branch

The output branch is the target data structure that the map uses to write JSON data. The map cannot use the output branch to validate JSON data. But, the output branch is available to the map to generate JSON data on output. The only JSON rules that the output branch provides are the initiators, terminators, and delimiters. The output branch, like the input branch, is divided into the following two parts:

  • Internals – contains the internal types to help with the output value type.
  • Types – contains the simplified JSON data types, including number, string, value, member, and holders for array and object definitions.
Figure 6. Output branch
Output branch contains: Internals and Types.

Validating JSON data with the generic validation map

To use a generic validation map to validate JSON data, follow these steps to run the map:

  1. Open the WTXJSON\WTX\PurchaseOrder.mms map source file.
  2. Open the GenericJSONValidation map.
  3. Review the input card to see how to use the generic JSON type tree to validate the WTXJSON\json\PurchaseOrder.json file. The Edit input card window displays the following values:
    • CardName = input
    • TypeTree = JSON.mtt
    • Type = Message JSON
    • FetchAs = Integral
    • WorkArea = Create
    • FetchUnit = S
    • Source = File
    • FilePath = ..\json\PurchaseOrder.json
    Figure 7. Input card
    Edit JSON input card (input) window.
  4. Build the GenericJSONValidation map.
  5. Run the GenericJSONValidation map to validate the PurchaseOrder.json file.

Transforming data with JSON data

The file, type tree layouts, and procedures in the following sections show how to transform data from a source COBOL copybook format to a destination JSON format. You use the generic JSON type tree to create a validation type tree. You then create the maps that validate the JSON data and transform the JSON data from a COBOL copybook format to a JSON format.

The source-format structure is described by a .cpy file and the destination-format structure is described by the JSON sample data. Both data structures, also referred to as formats, represent details of a purchase order with fields such as itemName, itemCount, and address. The following file layout presents the data descriptions of the related formats:

Listing 1. \WTXJSON\copybook\PurchaseOrder.cpy
000010     01  PURCHASE_ORDER
000020     02  ITEM 	  PIC X(30).
000030     02  ITEM_COUNT PIC X(10).
000040     02  NAME       PIC X(30).
000050     02  STREET     PIC X(30).
000060     02  CITY       PIC X(30).
000070     02  STATE      PIC X(30).
000080     02  ZIP        PIC X(10).
Listing 2. \WTXJSON\json\PurchaseOrder.json
{
        "item" : "book",
        "item_count" : 10,
        "shipTo" : {
                "name" : "John Doe",
                "street" : "123 Memorial Avenue",
                "city" : "Orlando",
                "state" : "FL",
                "zipCode" : "33455"
        }
}

Creating the PurchaseOrder JSON type tree

To create the custom PurchaseOrder JSON type tree to validate the JSON PurchaseOrder:

  1. Review all the name-value pairs in the data and group them by their JSON data type. In this example, the name-value pairs are:
    • JSON string (item, name, street, city, state, and zipCode)
    • JSON number (item_count)
    • JSON object (shipTo and purchaseOrder)
  2. Open the WTXJSON\json\PurchaseOrder.mtt type tree.
  3. To create the types for input validation, create a type for each name-value pair under the corresponding JSON data type presented in the Input type tree structure:
    • Basic
      • Number
        • item_count
      • String
        • city
        • item
        • name
        • state
        • street
        • zipCode
      • Value
    • Complex
      • Arrays
      • Member
      • Objects
        • purchaseOrder
        • shipTo
Figure 8. Input branch
Input branch includes: Internals and Types.
  1. To create the types for output generation, create a type for each name-value pair under the corresponding JSON data type presented in the type tree structure under Output:
    • Basic
      • Number
        • item_count
      • String
        • city
        • item
        • name
        • state
        • street
        • zipCode
      • Value
    • Complex
      • Arrays
      • Member
      • Objects
        • purchaseOrder
        • shipTo
Figure 9. Output branch
Output branch includes: Internals and Types.
  1. Set the following properties on the JSON types:
Table 1. JSON types properties table
JSON TypeInitiator (Literal, DataLanguage UTF-8)Terminator (Literal, DataLanguage UTF-8)Empty (Literal, DataLanguage UTF-8)
String"name"<OWSP>:<OWSP>"""name"<OWSP>:<OWSP>null
Number"name"<OWSP>:<OWSP>"name"<OWSP>:<OWSP>null
Boolean"name"<OWSP>:<OWSP>"name"<OWSP>:<OWSP>null
Object"name"<OWSP>:<OWSP>
{<OWSP>
<OWSP>}<OWSP>"name"<OWSP>:<OWSP>null
Array"name"<OWSP>:<OWSP>
[<OWSP>
<OWSP>]<OWSP>"name"<OWSP>:<OWSP>null
Top Level Object<OWSP>{<OWSP><OWSP>}<OWSP>N/A
Top Level Array<OWSP>[<OWSP><OWSP>]<OWSP>N/A
  1. Add the following components to the Input shipTo group:
    • name String Basic
    • street String Basic
    • city String Basic
    • state String Basic
    • zipCode String Basic
Figure 10. Components of Input shipTo group
Components of the shipTogroup.
  1. Add the following components to the Input purchaseOrder group:
    • item String Basic
    • item_count Number Basic
    • shipTo
Figure 11. Components of Input purchaseOrder group
Components of the purchaseOrder group.
  1. Add the following components to the Output shipTo group:
    • name String Basic
    • street String Basic
    • city String Basic
    • state String Basic
    • zipCode String Basic
Figure 12. Components of Output shipTo group
Components of the Output shipTo group.
  1. Add the following components to the Output purchaseOrder group:
    • item String Basic
    • item_count Number Basic
    • shipTo
Figure 13. Components of Output purchaseOrder group
Components of the Output purchaseOrder group.
  1. Validate and save the type tree. You now have a validation type tree with the generic JSON type tree.

Validating the PurchaseOrder JSON type tree

To create the PurchaseOrder type tree validation map:

  1. Open the WTXJSON\wtx\PurchaseOrder.mms map source file.
  2. Create a map that is called PurchaseOrderValidation.
  3. Create an input card that is called input with the same card settings as the following card:
    • CardName: input
    • TypeTree: ..\json\purchaseOrder.mtt
    • Type: purchaseOrder Objects Complex Types Input JSON
    • FetchAs: Integral
    • Source: File
    • FilePath: ..\json\PurchaseOrder.json
Figure 14. Input card input
Values for the JSON Input card.

To see how to use the PurchaseOrder JSON type tree to validate the data in the PurchaseOrder.json file, review the WTXJSON\json\PurchaseOrder.json file.

  1. Build the PurchaseOrderValidation map.
  2. Run the map to validate the PurchaseOrder.json file.

Transforming the JSON data format to PurchaseOrder copybook format

To create a map to transform the data in the PurchaseOrder JSON format to the copybook data format:

  1. Open the WTXJSON\wtx\PurchaseOrder.mms map source file.
  2. Create a map that is called PurchaseOrderJSON2Copybook.
  3. Create an input card that is called input with the same card settings as the following card:
    • CardName: input
    • TypeTree: ..\json\purchaseOrder.mtt
    • Type: purchaseOrder Objects Complex Types Input JSON
    • FetchAs: Integral
      • WorkArea: Create
      • FetchUnit: S
    • Source: File
    • FilePath: ..\json\PurchaseOrder.json
Figure 15. Input card input
Values for the JSON format Input card.
  1. Create an output card that is called output with the same card settings as the following output card:
    • CardName: output
    • TypeTree: ..\copybook\PurchaseOrder.mtt
    • Type: PURCHASE_ORDER Record CopyBook
    • Target: File
    • FilePath: ..\copybook\PurchaseOrderOutput.dat
Figure 16. Output card output
Values for the copybook format Output card.
  1. Create the following output card rules to map the data:
    • ITEM Field =TEXT(itemString Basic:input)
    • ITEM_COUNT Field =TEXT(item_count Number Basic:input)
    • Name Field =TEXT(name String Basic:shipTo:input)
    • STREET Field =TEXT(street String Basic:shipTo:input)
    • CITY Field =TEXT(city String Basic:shipTo:input)
    • STATE Field =TEXT(state String Basic:shipTo:input)
    • ZIP Field =TEXT (zipCode String Basic:shipTo:input)
Figure 17. Output card rules
Copybook Output cards rules.
  1. Build the PurchaseOrderJSON2Copybook map.
  2. Run the map.
  3. Open the WTXJSON\copybook\PurchaseOrderOutput.dat file and verify the output.

Transforming the PurchaseOrder copybook format to JSON data format

To create a map to transform the data in the PurchaseOrder copybook format to the JSON data format:

  1. Open the WTXJSON\wtx\PurchaseOrder.mms map source file.
  2. Create a map that is called PurchaseOrderCopybook2JSON.
  3. Create an input card that is called input with the same settings as the following input card:
    • CardName: input
    • TypeTree: ..\copybook\PurchaseOrder.mtt
    • Type: PURCHASE_ORDER Record CopyBook
    • FetchAs: Integral
      • WorkArea: Create
      • FetchUnit: S
    • Source: File
    • FilePath: ..\copy\PurchaseOrderOutput.dat
Figure 18. Input card input
Values for the copybook format input card.
  1. Create the following an output card that is called output:
    • CardName: output
    • TypeTree: ..\copybook\PurchaseOrder.mtt
    • Type: PURCHASE_ORDER Objects Complex Types Output JSON
    • Target: File
    • FilePath: ..\json\PurchaseOrderOutput.json
Figure 19. Output card output
Values for the JSON format Output card.
  1. Create the following rules to map the data:
    • output
      • item String Basic =ITEM Field:input
      • item_count Number Basic =ITEM_COUNT Field:input
    • shipTo
      • name String Basic =NAME Field:input
      • street String Basic =STREET Field:input
      • city String Basic =CITY Field:input
      • state String Basic =STATE Field:input
      • zipCode String Basic =ZIP Field:input
Figure 20. Output card rules
JSON format Output card rules listing.
  1. Build the PurchaseOrderCopybook2JSON map.
  2. Run the map.
  3. Open the WTXJSON\json\PurchaseOrderOutput.json file and verify the output.

NOTE: WebSphere Transformation Extender does not add white spaces or new lines to the output.


Conclusion

WebSphere Transformation Extender Design Studio maps data from any format that can be defined with type trees, XML schemas, or both. The example shows you how to use Design Studio to:

  • Define the structure of JSON data with a type tree
  • Transform binary data to JSON data
  • Transform JSON data to binary data

After you transform the data, you can deploy the data transformation with the runtime engine that is integrated within other IBM products such as: WebSphere DataPower, WebSphere Enterprise Service Bus, WebSphere Message Broker, WebSphere Business Process Manager, and IBM Sterling B2B Integrator.


Acknowledgements

The author would like to thank Robert Vila for the initial content from his article "Getting started with WebSphere Transformation Extender Design Studio on DataPower" (developerWorks, Dec 2007), upon which this article is based.


Downloads

DescriptionNameSize
Design Studio project artifactsWTXJSONSetup.zip10KB
Design Studio project solution artifactsWTXJSONSolution.zip12KB

Resources

Learn

Get products and technologies

  • Download IBM WebSphere products including WebSphere Transformation Extender Design Studio, version 8.4.0.2 or later.
  • Evaluate IBM products in the way that suits you best: Download a product trial, try a product online, use a product in a cloud environment, or spend a few hours in the SOA Sandbox learning how to implement Service Oriented Architecture efficiently.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Commerce on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Commerce, WebSphere
ArticleID=845088
ArticleTitle=Process JSON data with WebSphere Transformation Extender
publish-date=11082012