Build an SOA application using WebSphere Integration Developer V6.1.2 adapter patterns

Many Service Oriented Architecture (SOA) applications use adapters to access other systems, such as systems with FTP support, systems providing flat files with data for processing, and systems with e-mail messages for distribution across organizations. The adapter patterns in IBM® WebSphere® Integration Developer are a way of reducing development time when using adapters with your application. This article shows how to use adapter patterns to build an application that retrieves files from an FTP server and sends files to an FTP server. These patterns will make use of the IBM WebSphere Adapter for FTP. Since message transformation is a common function of applications that read and write files through the use of adapters, this article also illustrates how to create a module with a business object map to transform a message received from an input file and send it to an output file. An understanding of SOA applications and terminology is assumed. This content is part of the IBM WebSphere Developer Technical Journal.

Gary Bist (bist@ca.ibm.com), Technical Writer, IBM

Gary Bist is a technical writer at the IBM Toronto Lab who has worked on documentation in the adapters and other integration areas for several years. Gary also worked on documentation for the predecessor to WebSphere Integration Developer, WebSphere Studio Application Developer Integration Edition. You can reach Gary at bist@ca.ibm.com.



John Green (green@ca.ibm.com), Software Developer, Manager, IBM

John Green is a developer and manager at the IBM Toronto Lab with many years experience in adapter technology. John is a key contributor to the Enterprise Metadata Discovery specification, which allows adapters to be more readily brought in to integration development environments like WebSphere Integration Developer. You can reach John at green@ca.ibm.com.



Harold Gartner (hgartner@ca.ibm.com), Software Developer, IBM

Harold Gartner is a Software Developer at the IBM Toronto Lab who works on WebSphere Integration Developer and has worked on several other products that were the precursors to WebSphere Integration Developer. Harold has extensive development knowledge of the adapter wizards including the adapter patterns wizard. You can reach Harold at hgartner@ca.ibm.com.



30 July 2008

Introduction

An adapter pattern is a quick and easy way of creating a service that uses an adapter. Like other patterns in software, an adapter pattern is based on the way many users create a service that uses an adapter. An adapter pattern based on this common behavior can be seen as a type of shortcut to get up and running with an application that will work for many – but not all – users. In WebSphere Integration Developer, the following adapter patterns are available:

Table 1. WebSphere Integration Developer V6.1.2 adapter patterns
Name of adapter patternDescription
Inbound FTP adapter patternCreates a service that retrieves a file from a directory on an FTP server. If the file is not in an XML format, you can specify a data handler to transform the file content format to a business object, which is the format that is expected by SOA applications. The file content can be split if the content contains multiple instances of the data structure for processing.
Outbound FTP adapter patternCreates a service that retrieves a file from a directory on an FTP server. If the file is not in an XML format, you can specify a data handler to transform the file content format to a business object, which is the format that is expected by SOA applications. The file content can be split if the content contains multiple instances of the data structure for processing.
Inbound flat file adapter patternCreates a service that retrieves a file from a directory on a local file system. Similar to the FTP adapter pattern, a data handler can be used to transform the data if your organization requires a unique format. The file content can be split if the content contains multiple instances of the data structure for processing.
Outbound flat file adapter patternCreates a service that stores data in a file that is added to a directory on the local file system. A data handler can be used be used to transform the data, if the required data output format is unique to for your organization.
Outbound Email adapter patternCreates a service that sends simple messages using an Email server.

When you would use an adapter pattern

An adapter pattern is based on observing how most users would use a particular adapter. The result is a wizard which requires a minimum input from you since the values for many fields have been predetermined (based on what most users would have provided). In the case of the FTP adapter pattern, for example, you need to provide only the host name of the FTP server, the directories and the format of the input and output files.

If the simple and easy adapter pattern will work for most users, when will it not be appropriate? There are a few cases. Some generated services that use an adapter will never have an adapter pattern. These services need the regular adapter wizard because the wizard must interact with an EIS system and get metadata before generating the service. In other words, a pattern cannot be created because values cannot be predetermined. For example, an adapter that interacts with a database such as WebSphere Adapter for JDBC, requires that a database, tables and other database information be discovered before a service can be created.

Another example of when an adapter pattern will not work for the FTP adapter or the Flat File adapter is when you wish to use another method besides the Create or Append method in your application. When you create an outbound FTP service that does not use an adapter pattern, the following methods are created: Create, Append, Overwrite, Delete, Exists, List, Retrieve, ExecuteFTPScript and ServerToServerFileTransfer. The adapter pattern wizard only exposes the Create and Append method. The adapter pattern wizard, in a nutshell, covers 80% of the general use cases for the adapters. However, your case may be in the other 20%.


Overview of the application

Let’s look at the business problem we are trying to solve with our application. Our marketing team at the head office of a bank created a new financial service to offer customers. “We need to get information on this new financial service out to our customers right away! Can it be done?” asked the marketing director. The IT director recalled that most customers had email addresses in their customer records, which could be used for an electronic mail out. She added that using IBM’s new adapter pattern wizard it would be easy to extract the names and email addresses from the customer records kept at the branch offices. The branch offices simply had to put the customer records in a directory on an FTP machine.

Our application will extract the email addresses from the customer records at the bank’s branch offices. The branch offices will put the customer data in XML format in a file in a directory on an FTP machine. The application will pick up the XML file and reduce the information to just the customer name and email address. Our application will output this information to files that will be used to notify the bank’s customers through an email note that a new financial service is available.

The diagram below shows at runtime what the application will do. Let’s quickly walk through the technical aspects of the application before building the actual application with WebSphere Integration Developer.

The top rectangle represents a server at a branch office that has FTP support. An XML file with a list of customer records from that branch office is placed into a directory. Our application, which at this point has been set up using the FTP adapter pattern wizard, picks up the XML file using the WebSphere Adapter for FTP and reads in the customer data from that file. The adapter periodically polls the directory to check if another XML file has been added and again reads in the customer data for processing. At runtime, this is called inbound processing because the data is coming from an external source into the SOA application. The data streams into an export component of our application. An export is used to access data being sent from another external system.

Once read in, the application processes the data. A business object map and an interface map are used to take all the customer data and revise the collection of data to just the customer name and email address. That information is passed to an import component. An import is used to send data from an SOA application to an external system, in this case, back to the FTP server. An alternative design would be for the application to directly send the email since the application now has the customer name and email address isolated.

Figure 1. Describes the functions the application performs at runtime
Functions application performs at runtime

Build the application

Let’s get started. We will build the application in a process that is similar to the way most developers create applications. We will first set up our project files and then use the adapter patterns wizard to set up the interaction between the SOA application, the FTP adapter and the FTP server. Once this set up work is done, we will create a business object map and an interface map, used to separate just the customer’s name and email address from the entire set of customer data, and wire all the components together, which means linking all the SOA components together to create an SOA application. Lastly, we will test our application using WebSphere Integration Developer’s test environment. This means deploying our application to a built in server for test purposes, running the application and examining the output files.

Create a module

A module is similar in other products to a project folder. It contains all the elements of your application.

Follow these steps to create a module:

  1. From the menu bar, select File => New => Module. In the Module Name field, enter AdapterPatterns.
  2. Click Finish. The module appears in the Business Integration view.

Create two business objects for the module

Your module will require two business objects. One business object will contain the customer information that is read in from the bank’s branch. This business object will contain all the customer information (name, email address, street address, city and province.) The other business object will contain the customer information that will be returned to the bank’s branch. It will be used to market the bank’s new financial service. It will contain only the name and email address.

To create your business objects, follow these steps:

  1. Expand AdapterPatterns. Right-click Data Types. From the menu, select New => Business Object. In the Folder and Name fields, enter Customer. It is good programming practice to add a folder, as on generation the business object artifacts are grouped physically in folders rather than mixed with all the other files in the root folder. In our simple application, we used the same name. In a more complex application with many business objects, you would choose different names so you could group your business objects logically. Click Finish. The business object editor opens.
  2. Select Customer and then click Add a field to a business object, which you will find at the top of the editor.
  3. Change field1 to customerName. Similarly add the following fields: email, address, city and state. Leave the type as string. Save your work
  4. Create another business object called CustomerNameEmailOnly in the same manner with only customerName and email fields. Save your work.
  5. Close the business object editor when finished.
    Figure 2. Create the business objects
    Create the business objects
    Figure 3. Create CustomerNameEmailOnly business object
    Create CustomerNameEmailOnly

Setup file structure on FTP machine and local machine

Some directories need to be created that will be used by the application you develop in the AdapterPatterns module. An input file needs to be placed in one of them, which will represent the customer information that would be sent from one of the bank’s branch offices to the head office.

Create the following directories:

  1. On your server with FTP support, create two directories:
    FTPRead and FTPWrite Add customer.txt to FTPRead. This is the file that will be read in by the application. The contents of customer.txt follows these steps.
  2. Check that the FTP program on the server has given your userid permission to read, write and delete files.
  3. On your local computer where you have WebSphere Integration Developer, create a directory called ftp and add the following subdirectories to ftp:
    inboundarchive, outboundsequence and stagingdirectory
  4. You should have a file structure like the following:
       ftp\inboundarchive
          \outboundsequence
          \stagingdirectory

The flow of data in these directories is as follows. A file is placed in the FTPRead directory on the FTP machine. This file represents the customer data from a branch office of the bank. As each block of data representing a customer arrives in the local machine for processing, it is placed temporarily in a staging directory. While in this directory, a copy is placed in the archive directory with the suffix of a timestamp and a success or fail indicator. Success or fail is determined by the ActivationSpec class, which configures the transaction properties. You would only use an archive directory if you wanted to see if a file was successfully processed. From the staging directory, it is processed and the output sent to the FTPWrite directory.

The contents of customer.txt follow:

Listing 1. customer.txt
<?xml version="1.0" encoding="UTF-8"?>
<p:Customer xsi:type="p:Customer"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p=
         "http://AdapterPatterns/Customer">
  <customerName>John Doe</customerName>
  <email>johndoe@hayday.com</email>	  
  <address>170 Baseline Ave</address>
  <city>Ottawa</city>
  <state>ON</state>
</p:Customer>
####
<p:Customer xsi:type="p:Customer"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p=
         "http://AdapterPatterns/Customer">
  <customerName>Susan Kidman</customerName>
  <email>susankidman@cowtown.com</email>
  <address>104 Candlestick Park</address>
  <city>Ottawa</city>
  <state>ON</state>
</p:Customer>
####
<p:Customer xsi:type="p:Customer"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="
         http://AdapterPatterns/Customer">
  <customerName>Kathleen Black</customerName>
  <email>kathleenblack@ottawatower.com</email>  
  <address>530 Chanticler Road</address>
  <city>Ottawa</city>
  <state>ON</state>
</p:Customer>
####
<p:Customer xsi:type="p:Customer"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p=
         "http://AdapterPatterns/Customer">
  <customerName>Gary White</customerName>
  <email>garywhite@francaisplace.com</email>  
  <address>793 Morin Street</address>
  <city>Ottawa</city>
  <state>ON</state>
</p:Customer>
####

Create an inbound FTP service using the adapter pattern wizard

Create the inbound FTP service using the adapter pattern wizard with these steps.

  1. Launch the adapter pattern wizard. From the menu bar, select File => New => From Patterns. Expand Integration. Expand Adapters. Expand FTP. Select Create an inbound FTP service to read from a remote file. Click Next.
    Figure 4. inbound FTP service to read from a remote file
    inbound FTP service reads from a remote file
  2. In the FTP service name page, select AdapterPatterns for the module and add a service and folder name called CustomerInfoTorontoBranch. Adding a folder is recommended to contain the physical artifacts that will be generated. In our simple example, we used the same name for service and folder. However, in a large, complex application you would use different names since your folder would likely contain several services grouped logically. Click Next.
    Figure 5. Specify the name of the module and location of the inbound FTP service
    Module name and location of the inbound FTP service
  3. In the Business object and location page, select Customer as the business object. Identify your FTP server and specify your remote directory (FTPRead). Clicking Test connection checks for a connection to the FTP server. For staging directory, browse to the location of ftp\stagingdirectory that you created earlier. The staging directory is a directory for temporarily placing files that are being processed by the application.
    Note: In our example the staging directory on our local machine is identified with the Windows® notation for a file system; that is, backwards slashes to indicate directories. The remote directory, even though on a Windows’ server, uses the UNIX® convention of forward slashes for directories as the FTP program, not the operating system, determines the convention. Click Next.
    Figure 6. Specify the business object and location of the input file
    business object and location of the input file
  4. In the FTP server security credential page, add your Java™ Authentication and Authorization Services (JAAS) alias. The JAAS alias specifies the credentials for the FTP server (that is, the userid and password). The JAAS alias is set up on the WebSphere Process Server, not the FTP server. In other words, the JAAS alias is on the application server where the module will be deployed, not on the FTP server where the file that contains the customer information is created or fetched from. Your application uses the JAAS alias when running to retrieve the credentials stored on the WebSphere Process Server.
    Using the JAAS alias is the preferred method because no sensitive information is stored in a user’s application code. When you use the alternate method and specify a userid and password, this sensitive information is stored in plain text in the application code (that is, in the import or export file). Exposing sensitive information may be acceptable in testing situations but it is generally not acceptable in a production environment. Note that you can change from a userid and password to a JAAS alias after generating the service by changing the properties of the export in the assembly editor. Click Next.
    Figure 7. Specify the FTP server security credential
    FTP server security credential
  5. In the Input file format and file content split option page, select the file format. XML is used in our example and XML is the default. You could also use delimited, fixed width, JavaScript Object Notation (JSON). A comma-separated value is the default configuration for the delimited data handler.
    For the file content split option, we selected Split file content by delimiter. A splitter is used if there is more than one record in a file to be processed. For example if you are processing a spreadsheet with rows of records. Each record may be delimited by an end-of-line character. We entered a value of ####;\r\n. #### is the delimiter. ; indicates another delimiter follows. \r\n indicates an end-of-line and new line follows. An explanation of the delimiter values is found in the file splitting section of the WebSphere Adapter for FTP documentation. It will vary on different operating systems. Note that the default is to have no file splitting. Click Next.
    Figure 8. Specify the input file format and file content split option
    Input file format and file content split option
  6. In the Archive directory and wrapper business object page, browse to the inboundarchive directory you created earlier. The field will then contain the path and filename. At runtime, this directory will add a file indicating if the incoming file was processed successfully. You would use the archive directory if you want to keep track of the success or failure of files being processed. This track is created by copying the files to this directory and appending a timestamp and success or failure indicator to them.
    For our example, we did not choose to use a wrapper business object. You would use this option if your application needed additional information about the FTP adapter itself. Click Finish. An export is created in the assembly diagram editor; it represents your FTP inbound service. The values you entered when creating the service are shown when you select the export and select the properties view, where they may be modified. An FTP adapter project is added to your Business Integration view.
    Figure 9. Specify the archive directory and option to provide input file information
    Archive directory and option to provide input file information

Create an outbound FTP service using the adapter pattern wizard

Create the outbound FTP service using the adapter pattern wizard with these steps.

  1. Launch the adapter patterns wizard. From the menu bar, select File => New => From Patterns. Expand Integration. Expand Adapters. Expand FTP. Select Create an outbound FTP service to write to a remote file. Click Next.
    Figure 10. Select one of the patterns to create an artifact
    Patterns to create an artifac
  2. In the FTP service name page, add a service and folder name called CustomerNameEmailOnly. As stated previously, in our simple example, we used the same name for service and folder. However, in a large, complex application you would probably use different names since your folder might contain several services grouped logically. Click Next.
    Figure 11. Specify the name and location of the outbound FTP service
    Name and location of the outbound FTP service
  3. In the Business object and FTP server credentials page, select CustomerNameEmailOnly as the business object. Identify your FTP server and specify your remote directory (FTPWrite). Click Next.
    Figure 12. Specify a business object, FTP server host name and remote directory
    Business object, FTP server host name and remote directory
  4. In the FTP server security credential page, add your Java Authentication and Authorization Services (JAAS) alias for your FTP server. It is the recommended way of handling security, as explained previously in the notes accompanying the same page when creating an inbound FTP service. You may, however, use a userid and password. Click Next.
    Figure 13. Specify FTP server security credential
    FTP server security credential
  5. In the Outbound file name page, browse to the outboundsequence directory created earlier. Note that you must have a sequence file name or one will be created based on your business object name. At runtime, the sequence file in that directory indicates the processing order of the outgoing data. The other fields will be filled by the adapter patterns wizard based on your business object name.
    There are three methods for naming the output file. The first method is to generate a name using a sequence number. If you provide the name TorontoBranch.txt then the file name will be TorontoBranch.1.txt, TorontoBranch.2.txt, and so on for each customer’s data that is processed. A sequence file can be used for more than one output file. It is created on demand, and the adapter will place the name of the file and the next sequence number to use in it. The second method is to use the FTP server to generate a unique name. If you choose this option and need to know the name then it will be contained in the response business object. And finally you can specify the name of the file by using a wrapper business object. You would do this if the application is responsible for using a unique name and if the application will append further records to the file. Click Next.
    Figure 14. Specify the method for naming the output file
    Method for naming the output file
  6. In the Output file format page, accept the XML default format and click Finish. The import is created and shown in the assembly editor.
    Figure 15. Specify the format for the content of the output file
    Format for the content of the output file

Create a business object map

You want some of the customer data from the Customer business object to go the CustomerNameEmailOnly business object. A business object map will be the means of reducing the customer data to only the customer name and email address.

  1. In the Business Integration view, right-click Mapping and select New => Business Object Map. For the name and folder, enter CustomerToCustomerNameEmailOnly. Click Next.
  2. In the Business Object Map page, click Add beside the Inputs pane. In the Data Type Selection dialog, select Customer and click OK.
  3. Click Add beside the Outputs pane. In the Data Type Selection dialog, select CustomerNameEmailOnly and click OK. Click Finish.
  4. The business object mapping editor opens with the two business objects. Wire the customerName and email fields in the Customer business object to the same fields in the CustomerNameEmailOnly business object. Save your work and close the editor. br />
    Figure 16. Customer data from the Customer business object to go to CustomerNameEmailOnly
    Customer data from the Customer business object
    Figure 17. The business object mapping editor opens with the two business objects
    Editor opens with the two business objects

Create an interface map

You want to wire the export that contains the incoming customer data to the import that will return only the customer name and email address to the FTP server. This means you must create an interface map between the components and use the business object map you just created so that only the customer name and email address are passed on.

  1. In the Business Integration view, right-click Mapping and select New => Interface Map. In the name and folder fields, enter CustomerInfoToNameEmailOnly. Click Finish. The interface mapping editor opens.
  2. Click Add source Interface, which you will find at the top of the editor. In the Add source Interface dialog, select CustomerInfoTorontoBranch. Click OK.
  3. Click Add target Interface, which you will find at the top of the editor. In the Add target Interface dialog, select CustomerNameEmailOnly. Click OK.
  4. Wire the emitCustomer operation in the CustomerInfoTorontoBranch interface to the Create operation in the CustomerNameEmailOnly interface. The Parameter mappings section of the editor opens.
  5. Wire the emitCustomerInput parameter to the createInput parameter. Since it is only a one-way operation, you do not need to wire the createResponse parameter.
  6. Click Move, which is on the wire between these parameters. Click the Properties view tab. In the Parameter Mapping Type field, change the value to Map.
  7. Click the Details tab. In the Business Object Map field, select CustomerToCustomerNameEmailOnly. Save your interface map and close the editor.
    Figure 18. Create an interface map between the components
    Interface map between the components

Wire the inbound and outbound FTP services to the interface map

Wiring the interface map between the export and import completes the application.

  1. In the Business Integration view, drag the CustomerInfoToNameEmailOnly interface map onto the assembly editor between the import and export.
  2. Wire the CustomerInfoTorontoBranch export to the CustomerInfoToNameEmailOnly interface map. An Add Wire information message tells you the export interface will be added to the interface map. Click OK.
  3. Wire the CustomerInfoToNameEmailOnly interface map to the CustomerNameEmailOnly import.
  4. Save your work and close any open editors.
    Figure 19. The Parameter mappings section of the editor opens
    Parameter mappings section of the editor

Test the application

A test client allows you to test an SOA application before you deploy the application to a production environment. Testing consists of starting the test server and deploying your application to it. Once the application is deployed to the test server, the WebSphere Adapter for FTP will poll the directory on the FTP server and process any files found in the directory.

Start the server and deploy the application

To start the server, deploy your application and see the results of using the adapter patterns wizard, follow these steps.

  1. Click the Servers tab to view the test servers available. Right-click the WebSphere Process Server. From the context menu, select Start.
  2. When the Status column changes to Started, right-click the WebSphere Process Server again. This time select Add and Remove Projects from the menu. The AdapterPatternsApp will be listed beneath Available projects. Click Add to move it to Configured projects. Click Finish.
  3. Your application is now running.

Examine the output files

You will find the output files in the FTPWrite directory of your FTP server. Each file will contain the name and email of one of the customers in an input file plus some XML code. The output files will be named CustomerNameEmailOnly.<number>.txt. The content will be similar to the following:

Listing 2. output files
<?xml version="1.0" encoding="UTF-8"?>
<p:CustomerNameEmailOnly xsi:type="p:CustomerNameEmailOnly"   xmlns:xsi=
      "http://www.w3.org/2001/XMLSchema-instance" xmlns:p=
         "http://AdapterPatterns/CustomerNameEmailOnly">
  <customerName>Kathleen Black</customerName>
  <email>kathleenblack@ottawatower.com</email>
</p:CustomerNameEmailOnly>

The ftp\inboundarchive directory will contain a customer.txt<date><time>.success file showing a file processed successfully. The ftp\outboundsequence directory will contain a CustomerNameEmailOnly.seq file listing the sequence of the files placed in the FTPWrite directory.

Once you have finished examining the output files, remember to stop the WebSphere Process Server before closing WebSphere Integration Developer.


Conclusion

Using adapter patterns can save you development time when working on SOA applications that use adapters. In WebSphere Integration Developer, there are adapter patterns for the FTP, Flat Files and Email adapters. Although using adapter patterns may not be applicable in all cases, they are a shortcut to developing an SOA application when they can be used, as demonstrated in this article.

Resources

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Business process management on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Business process management, WebSphere
ArticleID=320101
ArticleTitle=Build an SOA application using WebSphere Integration Developer V6.1.2 adapter patterns
publish-date=07302008