Skip to main content

By clicking Submit, you agree to the developerWorks terms of use.

The first time you sign into developerWorks, a profile is created for you. Select information in your developerWorks profile is displayed to the public, but you may edit the information at any time. Your first name, last name (unless you choose to hide them), and display name will accompany the content that you post.

All information submitted is secure.

  • Close [x]

The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerworks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

By clicking Submit, you agree to the developerWorks terms of use.

All information submitted is secure.

  • Close [x]

Create custom operators for WebSphere DataStage

Blayne Chard, Software Engineer, IBM
Blayne Chard is a intern at the IBM Silicon Valley Lab in San Jose, Calif. He received is bachelor's degree with honors in Computer Science from Victoria University of Wellington, New Zealand. Blayne currently works for the WebSphere Information Server team.

Summary:  Create a simple DataStage operator, then learn how to load the operator into DataStage Designer. An operator is the basic building block of a DataStage job. Operators can read records from input streams, modify or use the data from the input stream, and then write the results to a output stream.

Date:  08 Feb 2007
Level:  Intermediate PDF:  A4 and Letter (1276 KB | 32 pages)Get Adobe® Reader®

Activity:  16852 views
Comments:  

Load the operator into DataStage

Set up the environment

As DataStage does not have access to the environment you set up for development, you have to make a few changes so that DataStage can find your operator. First, your operator's dll file is not inside Windows' or DataStage's PATH environment variable, the easiest way to put your operator into the PATH is to move the dll into the PXEngine's bin directory. Alternatively, you can update your windows PATH variable to include your project's directory.

However, the operator is still not found by DataStage, as there is no mapping from the dll to the OSH name. To fix this, open up the PXEngine's main operator.apt file, located in e:\IBM\InformationServer\Server\PXEngine\etc\, then add myhelloworld myhelloworld 1 to this file.

Add the operator

Before starting this section, you need to know some of the following information about your DataStage environment:

  • DataStage username
  • DataStage password
  • DataStage server name
  • DataStage server port
  • DataStage project
  1. Start DataStage Designer and log into a project that you are able to use.
  2. Once you are inside, right-click any folder in the repository view.
  3. Select New > Other > Parallel Custom Stage Type.



    Figure 2. Path to setting the stage type
    DataStage Designer: Path to setting the stage type

  4. Fill in the tabs as show in the following figures. You do not need to modify the mapping tab.

    Fill in the following under the General tab:

    1. Stage type name -- With the name of your operator (myhelloworld)
    2. Operator -- With the name of your operator (myhelloworld)
    3. Short description -- A short description of what the operator does (hello world operator)


    Figure 3. General tab
    General tab

    Fill in the following under the Links tab:

    1. Stream inputs Maximum -- With the maximum number of input links (1)
    2. Stream inputs Minimum -- With the minimum number of input links (1)
    3. Outputs Maximum -- With the maximum number of output links (1)
    4. Outputs Minimum -- With the minimum number of output links (1)



    Figure 4. Links tab
    Links tab

    Fill in your information in the appropriate text boxes under the Creator tab.



    Figure 5. Creator tab


    Add a new property under the Properties tab.

    1. Property name -- The name of the parameter as defined in the operator(uppercase)
    2. Data Type -- The data type (boolean)
    3. Default Value -- Default value to use (False)
    4. Required -- If the property is required (No)



    Figure 6. Properties tab
    Properties tab

  5. Click Ok. This brings up the Save as dialog box. Select an appropriate place to save, (the example uses the processing operator type folder) then click Save.



    Figure 7. Save as
    Save as

  6. You should now see a myhelloworld operator inside the Processing tab of the palette.



    Figure 8. Processing tab
    Processing tab

  7. To check that everything has completed successfully, build a simple parallel job to test the new custom operator. Create a job by dragging two sequential file operators and the myhelloworld operator onto the canvas, then link them together as shown in Figure 9.



    Figure 9. Link the operators together
    Link the operators together

  8. Open the two sequential file operators and set the details of the files they are reading from or writing to.



    Figure 10. Add the location of the file to read from
    Add the location of the file to read from



    Figure 11. Add the location of the file to write to
    Add the location of the file to write to

  9. Open the myhelloworld operator and set the uppercase parameter to True.



    Figure 12. Add "uppercase" to the list of parameters
    Add uppercase to the list of parameters



    Figure 13. Set uppercase to be true
    Set uppercase to be true

  10. Inside the myhelloworld operator, go to Input > Columns, and add the input column. Add a new column with the column name inCount and the SQL type Integer.



    Figure 14. Add input column information
    Add input column information

  11. Go to Output > Columns, and add the output columns. Add two new columns one with the column name outCount and the SQL type Integer, and another with the column name outHello and the SQL type Varchar.



    Figure 15. Add output column information
    Add output column information

  12. Save your job, then click the Compile icon.



    Figure 16. Click the Compile icon
    Click the compile icon

    This should result in the compile success dialog box.



    Figure 17. Compile successful dialog box
    Compile successful dialog box

  13. Click the Run icon, which spawns the run dialog box, where you click Run.



    Figure 18. Click the Run icon
    Click the Run icon



    Figure 19. Click Run
    Click Run

  14. If all goes well, in a couple of seconds the links should turn green in your canvas and display the number of records they have processed.



    Figure 20. Completed process
    Completed process

Congratulations! You now have a working MyHelloWorld operator inside DataStage.

5 of 8 | Previous | Next

Comments



Help: Update or add to My dW interests

What's this?

This little timesaver lets you update your My developerWorks profile with just one click! The general subject of this content (AIX and UNIX, Information Management, Lotus, Rational, Tivoli, WebSphere, Java, Linux, Open source, SOA and Web services, Web development, or XML) will be added to the interests section of your profile, if it's not there already. You only need to be logged in to My developerWorks.

And what's the point of adding your interests to your profile? That's how you find other users with the same interests as yours, and see what they're reading and contributing to the community. Your interests also help us recommend relevant developerWorks content to you.

View your My developerWorks profile

Return from help

Help: Remove from My dW interests

What's this?

Removing this interest does not alter your profile, but rather removes this piece of content from a list of all content for which you've indicated interest. In a future enhancement to My developerWorks, you'll be able to see a record of that content.

View your My developerWorks profile

Return from help

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Information Management, WebSphere
ArticleID=194415
TutorialTitle=Create custom operators for WebSphere DataStage
publish-date=02082007
author1-email=bchard@us.ibm.com
author1-email-cc=

Tags

Help
Use the search field to find all types of content in My developerWorks with that tag.

Use the slider bar to see more or fewer tags.

Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).

My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).

Use the search field to find all types of content in My developerWorks with that tag. Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere). My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).

Try IBM PureSystems. No charge.