Skip to main content

Resource monitoring in WebSphere MQ File Transfer Edition

Laurence Bonney, WebSphere MQ JMS Test Team Lead, IBM
Laurence Bonney is a Software Engineer on the WebSphere MQ Development Team at the IBM Hursley Software Lab in the UK. You can contact Laurence at bonneyl@uk.ibm.com.

Summary:  WebSphere MQ File Transfer Edition (hereafter called WebSphere MQ FTE) provides reliable, auditable, and managed transfer of files, regardless of their size, between IT systems without the need for programming. This article takes you through a number of scenarios that show you how to use the WebSphere MQ FTE resource monitoring function.

Date:  21 Oct 2009
Level:  Intermediate
Activity:  1672 views

Resource monitoring overview

The resource monitoring functionality in IBM® WebSphere® MQ FTE was added to the product in V7.1 to provide a more flexible way to initiate tasks. The new Ant functionality enables you to initiate a file transfers and a wide array of other tasks. This article focuses on file transfers, and the scenarios below describe the movement of files as the outcome of a resource monitors trigger condition being met.

In its most generic form, resource monitoring lets you monitor a resource and initiate a predefined task when a trigger condition has been met. In WebSphere MQ FTE V7.0.1, the scope of the monitored resource is a directory or a number of subdirectories. The trigger condition is the arrival, update, or removal of a file within the monitored directories.

This article describes two scenarios. All examples in the scenarios are based on the command-line commands supplied with WebSphere MQ FTE, rather than using the MQ Explorer WMQFTE Plug-in. However, the concepts detailed in the article are easily transferred. When performing the tasks below, you need to have correctly configured your MQ network and defined and started the specified agents. As long as the agents can contact each other over the MQ network, it is of no consequence whether they reside on the same machine or use the same queue manager or different ones. The scenarios were created on a single Windows machine, but the same concepts apply to any platform that WebSphere MQ FTE supports.The two scenarios are:

Simple resource monitoring

This scenario involves a source agent and a destination agent. The agent queue managers have the same name as the agents. You will create a resource monitor defined on the source agent (as of WebSphere MQ FTE V7.0.1, the resource monitor must always be defined on the source agent and therefore the directory to monitor must be accessible by the source agent). The resource monitor will look for the existence or update of a file named trigger.go in the monitored directory C:\monitored. When the trigger condition is met, the defined task is to transfer the file C:\source\sourcefile.txt from the source agent to C:\destination\destinationfile.txt via the destination agent.

Creating the task file

First, create a file named task.xml that defines the transfer task to initiate when the monitor trigger is met. Use the -gt option on fteCreateTransfer, which, rather than initiating a transfer, writes the XML that defines the transfer to a file specified as a parameter to the -gt option. This XML can be manually put onto the source agent's command queue, or in this case, can be used with fteCreateMonitor:


Listing 1. Creating a simple resource monitor task file

C:\WMQFTE\bin> fteCreateTransfer -sa SOURCEAGENT –sm SOURCEAGENT -da DESTINATIONAGENT
–dm DESTINATIONAGENT -de overwrite -t text -df
C:\destination\destinationfile.txt -gt C:\monitored\task.xml 
C:\source\sourcefile.txt 5655-U80, 5724-R10 Copyright IBM Corp.  2008, 2009.  
ALL RIGHTS RESERVED BFGCL0204I: Transfer request has been output to 
C:\monitored\task.xml

The task XML file can be subsequently edited before use, which will be covered in the next scenario. However for this simple transfer, the generated task file is sufficient.

Creating the resource monitor

With the task defined, you can now create the resource monitor. In this example, define the new monitor SimpleResourceMonitor on the source agent SOURCEAGENT, monitoring the directory C:\monitored for the existence or update of the file trigger1.go:


Listing 2. Creating a simple resource monitor

C:\WMQFTE\bin> fteCreateMonitor -ma SOURCEAGENT -mn SimpleResourceMonitor -md 
C:\monitored -mt C:\monitored\task.xml -tr match,trigger.go -jn SimpleMonitorJob 
5655-U80, 5724-R10 Copyright IBM Corp.
2008, 2009.  ALL RIGHTS RESERVED BFGCL0188I: The request to create a monitor has
been submitted with a request id of 414d5120434f4d4d414e442020202020f3b48b4a20001f03.
BFGCL0251I: The request has successfully completed.

Executing fteListMonitors with verbose output shows the defined monitor:


Listing 3. Displaying a simple resource monitor

C:\WMQFTE\bin>fteListMonitors -v
5655-U80, 5724-R10 Copyright IBM Corp.  2008, 2009.  ALL RIGHTS RESERVED 
Monitor Information:
    Name:             SIMPLERESOURCEMONITOR
    Agent:            SOURCEAGENT
    Status:           Started
    Root directory:   C:\monitored
    Condition:        Match
    Pattern:          trigger.go

Testing the resource monitor

You now have two agents and a resource monitor actively polling the directory C:\monitored for the existence of the file trigger.go. At this point, make sure that C:\source\sourcefile.txt exists on the source agent's file system, and that C:\destination\ exists on the destination agent's file system. Now creating the file trigger.go in C:\monitored will cause the monitor to initiate the defined transfer task within the default polling interval of one minute. You can check the outcome either by looking for the file C:\destination\destinationfile.txt, which should now exist on the destination agent's file system, or via the WebSphere MQ Explorer GUI WMQFTE Plug-in, which should show a successful transfer of the file between the two agents. If the trigger file is updated so that the modified date is changed, the monitor will trigger again within the one minute polling interval and will transfer the file again. Since the original task has the -de overwrite option, even though the destination file already exists from the previous transfer, the new transfer will complete successfully.

Resource monitoring with variable substitution

This scenario involves four agents: one source agent and three destination agents named AccountsAgent, FinanceAgent, and HRAgent. As in the first scenario, the agent queue managers have the same names as the agents, and you create a resource monitor defined on the source agent. The resource monitor looks for the existence or update of any files with the extension .txt in the monitored directory C:\monitored or any or its subdirectories up to a maximum of one level of recursion. When the trigger condition is met, the defined task executes, transferring the trigger file from the source agent to the destination agent defined by the subdirectory name that the trigger file resides in. Additionally, it will rename the destination file based on the source file name, changing the .txt extension to .<AGENTNAME>, where <AGENTNAME> is the name of the destination agent that the file is being transferred to. This scenario shows how you can transform a static task file into dynamic file transfers using the variable substitution function available in the resource monitor.

Creating the task file

As with the basic scenario, you first need a task file to work from. Since the task file will need to be manually edited, you can use the task file generated from the previous scenario (see Listing 1. Creating a simple resource monitor task file. The task file should initially look like this:


Listing 4. Basic Task File
 
<?xml version="1.0" encoding="UTF-8"?> 
<request version="2.00"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="FileTransfer.xsd">
  <managedTransfer>
    <originator>
        <hostName>localhost</hostName> 
        <userID>Laurence</userID>
    </originator> 
    <sourceAgent qmgr=”SOURCEAGENT” agent="SOURCEAGENT"/>
    <destinationAgent qmgr=”DESTINATIONAGENT” agent="DESTINATIONAGENT"/>
    <transferSet>
      <item mode="text" checksumMethod="MD5">
        <source recursive="false" disposition="leave">
          <file>C:\source\sourcefile.txt</file>
        </source> 
        <destination type="file" exist="overwrite">
          <file>C:\destination\destinationfile.txt</file>
        </destination>
      </item>
    </transferSet>
  </managedTransfer>
</request>

Based on the scenario description, the following items are considered dynamic in this scenario.


Sections to be variable substituted
DescriptionOld static valueNew dynamic value
Source filenameC:\source\sourcefile.txt${FilePath}
Destination AgentDESTINATIONAGENT${FilePath{token=-2}{separator=/}}
Destination QmgrDESTINATIONAGENT${FilePath{token=-2}{separator=/}}
Destination filenameC:\destination\destinationfile.txtC:/destination/${FileName{token=1}{separator=.}}.${FilePath{token=-2}{separator=/}}

${FilePath} will be read by the resource monitor and substituted at task execution time by the value it currently has for the full path, including the filename of the trigger file. From this single variable, you can perform a number of manipulations to make some interesting transfers. For information on a number of additional variables that you can use in a similar way, see the WebSphere MQ FTE V7.0.1 Information Center. For all the references to a directory separator, the Unix style directory separator / is used. Since it is compatible for both Windows and Unix, it keeps the task file somewhat platform generic.

${FilePath{token=-2}{separator=/}} is read by the resource monitor using the value as with ${FilePath}. However, in this instance you split the value up using the defined separator /, then take the second token working right to left and substitute that value in. In the case of this scenario, that will be the subdirectory name under the directory C:\monitored\. For example, in this variable substitution, the trigger file C:\monitored\FinanceAgent\reports.txt will become FinanceAgent. For more information on the available manipulation methods, see the WebSphere MQ FTE V7.0.1 Information Center.

C:\destination\${FileName{token=1}{separator=.}}.${FilePath{token=-2}{separator=/}} is read by the resource monitor and substituted at task execution time. In this case you use both ${FilePath} and ${FileName} substitution variables. ${FileName} evaluates to the name of the trigger file ignoring any paths. In this instance ${FileName{token=1}{separator=.}} will evaluate to the first token split by the character "." effectively keeping the file name but removing the extension. As the token is specified as a positive rather than negative value, the token taken is worked from left to right. Then ${FilePath{token=-2}{separator=/}} evaluates to the same as the Destination Agent previously. So for example in this variable substitution, the trigger file C:\monitored\FinanceAgent\reports.txt, will become C:\destination\reports.FinanceAgent.

By replacing the old static values with these new dynamic values, all three can be different each time the monitor finds a match and triggers a transfer, purely based on the name of the trigger file matched and its location in the subdirectories. The task file should now look like this:


Listing 5. Variable substituted task tile

<?xml version="1.0" encoding="UTF-8"?> 
<request version="2.00"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="FileTransfer.xsd">
  <managedTransfer>
    <originator>
        <hostName>localhost</hostName> 
        <userID>Laurence</userID>
    </originator> <sourceAgent QMgr="SOURCEAGENT" agent="SOURCEAGENT"/>
    <destinationAgent QMgr="${FilePath{token=-2}{separator=/}}"
        agent="${FilePath{token=-2}{separator=/}}"/>
    <transferSet>
      <item mode="text" checksumMethod="MD5">
        <source recursive="false" disposition="leave">
          <file>${FilePath}</file>
        </source>
          <destination type="file" exist="overwrite">
          <file>C:/destination/${FileName{token=1}{separator=.}}.
               ${FilePath{token=-2}{separator=/}}</file>
        </destination>
      </item>
    </transferSet>
  </managedTransfer>
</request>

Creating the resource monitor

Now that a task file has been modified to include the variable substitution notation required for this scenario, you can define the resource monitor. In this example, you define the new monitor VarSubResourceMonitor on the source agent SOURCEAGENT, monitoring directory C:\monitored for the existence or update of any file with the extension .txt:


Listing 6. Creating variable substitution resource monitor
 

C:\WMQFTE\bin> fteCreateMonitor -ma SourceAgent 
-mn VarSubResourceMonitor -md C:\monitored -mt C:\monitored\task.xml 
-tr match,*.txt -rl 1 -jn VariableSubstitutionMonitorJob 
5655-U80, 5724-R10 Copyright IBM Corp. 2008, 2009.  ALL RIGHTS RESERVED 
BFGCL0188I: The request to create a monitor has been submitted with a request id of 
414d5120434f4d4d414e442020202020f3b48b4a20002203.
BFGCL0251I: The request has successfully completed.

Executing fteListMonitors with verbose output shows the defined monitor:


Listing 7. Displaying variable substitution resource monitor

C:\WMQFTE\bin>fteListMonitors -v 
5655-U80, 5724-R10 Copyright IBM Corp.  2008, 2009.  ALL RIGHTS RESERVED Monitor
Information: 
Name:             VARSUBRESOURCEMONITOR 
Agent:            SOURCEAGENT 
Status:           Started
Root directory:   C:\monitored 
Condition:        Match
Pattern:          *.txt

Testing the resource monitor

You now have four agents, and a resource monitor actively polling the directory C:\monitored and the first level of its subdirectories for the existence of any files with the extension .txt. Make sure the directory C:\destination\ exists on the destination agent's file systems. If you now create a file named C:\monitored\FinanceAgent\results.txt in the source file system, the monitor will initiate the defined transfer task within the default polling interval of one minute. You should see a file appear in the C:\destination directory of the FinanceAgent’s file system named results.FinanceAgent. As with the previous scenario, you can check the outcome of the transfer using the MQ Explorer WMQFTE Plug-in, which should show a successful transfer of the file between the two agents. Repeating this for C:\monitored\HRAgent\results.txt, and C:\monitored\AccountsAgent\results.txt should yield similar results for those agents.

Troubleshooting

Although not specific to resource monitoring, when transfers do not occur when expected, or monitors do not show as registered when defined, you can do a number of things to find out what happened:

  • Check the agent's Output0.log files for any information on the action that you expected to happen.
  • Subscribe to the SYSTEM.FTE topic to pick up the audit and log message output by the agents: In the MQ Explorer, right-click on the SYSTEM.FTE topic on the coordination Queue Manager and select Test Subscription. Specify the topic string SYSTEM.FTE/# to get all messages sent to the SYSTEM.FTE topic.
  • Check the agent queues and transmit queues for messages building up, which may indicate an MQ networking problem.

For more information on troubleshooting, see the WebSphere MQ FTE V7.0.1 Information Center.

Conclusion

This article has described a number of applications in which resource monitoring can be beneficial. Resource monitoring opens up numerous uses of WebSphere MQ FTE that were not available prior to V7.0.1. The scenarios have by no means covered all variations and uses of resource monitoring, but should give you an indication of the power of resource monitoring within WebSphere MQ FTE.


Resources

About the author

Laurence Bonney is a Software Engineer on the WebSphere MQ Development Team at the IBM Hursley Software Lab in the UK. You can contact Laurence at bonneyl@uk.ibm.com.

Comments (Undergoing maintenance)

Back to top

Trademarks  |  My developerWorks terms and conditions

Close [x]

Help: Update or add to My dW interests

What's this?

This little timesaver lets you update your My developerWorks profile with just one click! The general subject of this content (AIX and UNIX, Information Management, Lotus, Rational, Tivoli, WebSphere, Java, Linux, Open source, SOA and Web services, Web development, or XML) will be added to the interests section of your profile, if it's not there already. You only need to be logged in to My developerWorks.

And what's the point of adding your interests to your profile? That's how you find other users with the same interests as yours, and see what they're reading and contributing to the community. Your interests also help us recommend relevant developerWorks content to you.

View your My developerWorks profile

Return from help

Close [x]

Help: Remove from My dW interests

What's this?

Removing this interest does not alter your profile, but rather removes this piece of content from a list of all content for which you've indicated interest. In a future enhancement to My developerWorks, you'll be able to see a record of that content.

View your My developerWorks profile

Return from help

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere
ArticleID=438764
ArticleTitle=Resource monitoring in WebSphere MQ File Transfer Edition
publish-date=10212009
author1-email=bonneyl@uk.ibm.com
author1-email-cc=

Table of contents

Next steps from IBM

My developerWorks community

Interact, share, and communicate with developers worldwide.

My developerWorks overview

Tags

Help
Use the search field to find all types of content in My developerWorks with that tag.

Use the slider bar to see more or fewer tags.

Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).

My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).

Use the search field to find all types of content in My developerWorks with that tag. Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere). My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).

 

Dig deeper into WebSphere on developerWorks

Rate a product. Write a review.

Rate a product. Write a review.

Write a review for Rational products today!

Special offers

Rate and Review Rational products

WebSphere Application Server Hypervisor trial

SOA Tutorials IBM Book + DVD 35% off: IBM0402

More offers