Introduction

On my very first WebSphere MQ File Transfer Edition deployment, my client wished to run WebSphere MQ File Transfer Edition client agents on unattended Windows servers. Although WebSphere MQ File Transfer Edition does not provide a Windows service today, it wasn't hard to make the agents run as background tasks. The real trick was making them resilient. The fteStartAgent command forks a child process and then dies. This means that if configured as a Windows service, it either cannot be restarted or it tries to restart at every interval. We were able to directly start the agent by bypassing fteStartAgent and calling the underlying class directly, but were wary of using anything other than the documented command.

The solution we decided to use is the subject of this article. It is comprised of two parts: the Windows Service Trigger Monitor program provided as SupportPac MA7K, and a small bit of “glue” code written in Perl. Not only does it start the WebSphere MQ File Transfer Edition agent as desired, it actually makes the agent very difficult to kill.

The core component is SupportPac MA7K, which is dependent on the WebSphere MQ Client. The WebSphere MQ Client is available as SupportPac MQC7 (see Resources for information on both SupportPacs). The MA7K service connects to a queue manager and listens on what is called an initiation queue. The queue manager is configured to place a message on this queue in response to a message arriving in the WebSphere MQ File Transfer Edition agent's command queue. On receipt of a trigger message, the trigger monitor parses the message for the command to be executed and any parameters. These are assembled into a command line which is run, and then control returns to the trigger monitor which resumes listening on its initiation queue.

The Perl code is necessary because the trigger monitor assumes the program to be started has been written for triggering and passes the entire TMC2 trigger message as a parameter. Of course, the fteStartAgent program was written for humans, not for trigger monitors, and has no concept of what a TMC2 structure looks like. It therefore fails to run if started directly by the trigger monitor. A small Perl program serves as a "shim" between fteStartAgent and the trigger monitor, consuming the TMC2 parameter and converting it to fteStartAgent native parameter values. Perl was chosen for its string handling capabilities and because it is installed or readily available on all versions of Windows servers. It can also be compiled into a native Windows executable which runs without any need to install Perl.

The final step is to configure the queue manager for triggering. This includes definition of a process object and to enable triggering on the agent command queue. The process definition includes the fully-qualified command to start an agent, as well as the name of the agent to be started.

After we installed the first agent trigger monitor on this project, we discovered some additional benefits. A single trigger monitor can handle any number of agents on the same Windows server. Use of the trigger monitor also means that the WebSphere MQ administrator can start or stop the remote client agent from the queue manager and without any need to sign onto the Windows server where the agent runs. One of the issues that my customer faced was that the WebSphere MQ administrators did not have direct access to the Windows servers, so the ability to remotely start and stop agents was particularly useful.

Below are the steps to help you set up your own Windows client trigger monitor for starting WebSphere MQ File Transfer Edition agents. These instructions assume that the WebSphere MQ File Transfer Edition client agent has been installed to C:\IBM\WMQFTE and that the configuration directory is C:\IBM\WMQFTE\config. I have found that it is much easier to administer WebSphere MQ File Transfer Edition on Windows if I use path names that are short and have no embedded spaces. The account that runs the trigger monitor service should have C:\IBM\WMQFTE\bin and the Perl runtime in its path. It is best to configure the WebSphere MQ File Transfer Edition agent and ensure that it runs correctly before beginning the process of installing the instrumentation described here.

Back to top

Set up and test a Windows client trigger monitor

1. Install fteTriggerAgent.pl

   Download the fteTriggerAgent.pl file (see Resources or refer to Listing 3, below) and place it in the C:\IBM\WMQFTE\bin directory on the server where the WebSphere MQ File Transfer Edition agent will run.

   The script expects WebSphere MQ File Transfer Edition to be installed in C:\IBM\WMQFTE\ and it will attempt to write log files to C:\IBM\WMQFTE\Logs. If the path does not exist, the script will attempt to create it. If you want the log files from the fteTriggerAgent.pl script to be written to a different directory, you will need to change the $LogPath variable that is located near the top of the script.

   The script file should have the same ownership and permissions as the rest of the files in C:\IBM\WMQFTE\bin. To run the script, the .pl extension should be associated with the Perl executable. (If you wish to compile the script into an executable, take a look at Perl Pro Studio from Active State.)

2. Install WebSphere MQ client

   If not already installed on the server where the WebSphere MQ File Transfer Edition agent will run, obtain the latest WebSphere MQ client software. Currently, this is SupportPac MQC7, which can be found on the main SupportPacs page (see Resources). Sign on as an administrator and install the software as described in the WebSphere MQ Quick Beginnings for Windows documentation.

   Check the Recommended Fixes for WebSphere MQ page to see if a fix pack exists at a later version than the client software you just downloaded. If so, download and install it according to the instructions provided with the fix pack.

3. Install SupportPac MA7K

   Download SupportPac MA7K from the main SupportPacs page.

   Unzip the files onto the Windows server where the WebSphere MQ File Transfer Edition agent will run. The preferred location is adjacent to the WMQFTE directory under the IBM parent directory. For example, if WebSphere MQ File Transfer Edition was installed to C:\IBM\WMQFTE, then install MA7K to C:\IBM\MA7K. This is not strictly required but it makes things easier for administrators to locate these related products in adjacent directories.

   The trigger monitor service is installed using a setup program that reads the configuration from an .ini file. Edit the setup.ini file to provide details for the service account, the connection and the initiation queue. Listing 1 shows a sample.

   
   
   Listing 1
   

   Global: ShortTmr=60 ShortRty=10 LongTmr=1200 LongRty=999999999 EventLevel=2 WaitInterval=60000 ** Note that the file is a kdb format and the extension is not specified KeyRepository=C:\IBM\WMQFTE\config\<coordination qmgr>\ssl\key ** Specify the mqfte service userid (in the form: domain\user) ** The setup program will prompt for a password. ** If the user is invalid, you will receive a "Error number 1057" at setup time ** Ensure the user has rights to logon as a service ServiceUserid=.\mqfte MQSeriesDLL=mqic32.dll ** For each thread to run, there is a "thread" stanza, maximum 16 stanzas Thread: TriggerQueueName=SYSTEM.FTE.INITQ.<hostname> TriggerQueueMgrName=<agent QMgr> CONNAME=<hostname(port)> CHANNEL=FTE.SSL.SVRCONN SSLCIPH=TRIPLE_DES_SHA_US

   
   

   Before proceeding, let's look at a few of the items in the setup.ini file:

   • The service user ID is in the form domain\user. In the setup.ini file, the value ".\mqfte" means to use the mqfte account defined on the local host. If you use a different service account, be sure to update this line in the setup.ini file.
   • The service is set up to use SSL channels in the demo. By all means, please use SSL for agents and the agent trigger monitor. For initial testing, however, you can remove SSL errors from the equation if necessary by disabling SSL. Be sure to re-enable it later.
   • Because the trigger monitor is a C program and the agent is Java, they use different keystore types. You can convert the agent's Java keystore to a kdb keystore and use the same certificate, or you can use a different certificate for the trigger monitor. In this case, the keystore is a standard kdb file called key.kdb and a stash file names key.sth.
   • I chose to use an initiation queue per host instead of per agent. This permits one trigger monitor on a given host to serve multiple WebSphere MQ File Transfer Edition agents on that host.
4. Install the trigger monitor service.

   From a command prompt, navigate to the directory where MA7K was installed and where the setup.ini file resides. Install the service by running the setup command and specifying the setup.ini file:

   setup -f setup.ini

   Enter the password for the mqfte service account when prompted. (Be aware that "mqfte" is a fictional service account and there is nothing special about the name. You can use any service account name as long as it does not have administrator rights. Remember, the trigger monitor will execute any command passed to it from the initiation queue so it is necessary to limit who can put messages to the initiation queue and to run the trigger monitor as a non-administrative user.)

   After running setup, verify that the service is present in the Services applet (Figure 1). The service will install in Automatic Start mode, meaning that Windows will attempt to start the service at boot time, which is the desired behavior. However, the setup program itself does not attempt to start the service, so although it should be visible in the services panel, the status column for the service will be empty.

   
   
   Figure 1. Install trigger monitor service
   
   
   

   Start the service and wait a few moments while it initializes. Any errors related to starting and running the service will result in a dialog stating that the service has failed. Most conditions that can cause these events are caught in the setup process. On the other hand, the service might have problems connecting to the queue manager or accessing its initiation queue. These run time errors are reported in the Windows event log. If there are no errors reported starting the service applet, check the Windows Event Log. At this stage, you will see errors indicating that either the channel or the initiation queue do not exist. This is as expected and those objects will be created in the next section. For now just stop the service.

   Once the service is confirmed to start correctly, modify the service definition so that the service is automatically restarted. Set first, second, and subsequent actions to restart and the interval to 10 minutes, as shown in Figures 2, 3, and 4.

   
   
   Figure 2. General service startup
   
   
   
   
   Figure 3. Logon tab
   
   
   
   
   Figure 4. Recovery tab
   
   
   
5. Configure queue manager

   A single trigger monitor can serve any number of agents on the same host, as long as they use the same Agent QMgr. On the queue manager, this translates to a requirement for an initiation queue for each host on which an agent resides. For this reason, I chose a naming convention for initiation queues of SYSTEM.FTE.INITQ.<hostname> where the hostname is in UPPER CASE and 22 characters or less. You can use any queue name you wish for the initiation queue but I strongly advise against using SYSTEM.DEFAULT.INITIATION.QUEUE for security reasons.

   It is worth mentioning that picking a queue name of SYSTEM.FTE.INITQ.* could possibly conflict with future queue names used by the product. As a general rule, picking object names of SYSTEM.* means you must watch for conflicts over time as you upgrade to new product versions. If you want to avoid this problem altogether, choose a different name for your initiation queue that does not begin with SYSTEM. Just be sure to update the references to the queue name in the trigger monitor's setup.ini file and the agent's command queue.

   Because there is one process object per agent, the naming convention I chose embeds the agent name into the process name. Process definitions are named SYSTEM.FTE.<agent name> where agent name matches the case and spelling of the agent name as used in the SYSTEM.FTE.COMMAND.<agent name> queue. The same advice about picking names with SYSTEM.* in them applies here. It is probably not a big risk but if it concerns you, choose a different name.

   Enabling triggering on the queue manager requires definition of two objects and altering the existing command queue, as shown in Listing 2.

   
   
   Listing 2
   

   DEFINE QLOCAL(SYSTEM.FTE.INITQ.<hostname>) + DESCR('FTE initiation queue for <hostname>') + REPLACE DEFINE PROCESS(SYSTEM.FTE.<agent name>) + APPLTYPE(WINDOWSNT) + APPLICID('C:\IBM\WMQFTE\bin\fteTriggerAgent.pl -v') + ENVRDATA('C:\IBM\WMQFTE\bin\fteStartAgent.cmd') + USERDATA('<agent name>') +  DESCR('Trigger process for <agent name>') + REPLACE ALTER QLOCAL(SYSTEM.FTE.COMMAND.<agent name>) + INITQ(SYSTEM.FTE.INITQ.<hostname>) + PRO(SYSTEM.FTE.<agent name>) + TRIGGER + TRIGTYPE(FIRST)

   
   

   Remember, the trigger will only fire when:

   • There is no open input handle on the agent command queue,
   • and there is an open input handle on the initiation queue,
   • and queue depth in the agent command queue changes from zero to one.

   That last part is the tricky part. If there are messages in the agent command queue when you start the trigger monitor, the trigger will not fire. Or, more correctly, it will not fire until TRIGINT milliseconds have elapsed, and TRIGINT defaults to 999,999,999, which is about 278 hours. To insure your agent is triggered promptly, either clear the queue before starting the trigger monitor or set TRIGINT to a more practical interval.

6. Test driving your new trigger monitor service

   The MA7K trigger monitor service makes the WebSphere MQ File Transfer Edition agents extremely resilient. The fteStopAgent command will still work but the trigger monitor will automatically restart the agent immediately because the WebSphere MQ File Transfer Edition agent puts a placeholder message in its command queue as it shuts down. As soon as the agent releases the input handle on its command queue, the queue manager detects that the command queue has a depth greater than zero, fires a new trigger event, and the agent restarts. Try sending fteStopAgent a few times to get a feel for how it works. You will have to be quick if you want to catch the agent while it is down because the trigger monitor restarts it almost immediately. If in doubt, check the agent log files and you will see it stopping and restarting.

   When you wish to intentionally stop the WebSphere MQ File Transfer Edition agent, it is necessary to first disable triggering on the agent's command queue and then run the fteStopAgent command. To restart the agent remotely, just re-enable triggering on the agent's command queue. You can then wait and let it start when a command arrives or you can send it any command, such as ftePingAgent or even fteStopAgent (which places a Stop command on the agent's command queue, thus causing the trigger to fire) and the trigger monitor will start the agent.

Back to top

Conclusion

The MA7K Windows trigger monitor service is a great way to automatically start WebSphere MQ File Transfer Edition client agents on Windows servers at boot time. In addition to starting the agent, the trigger monitor restarts the agent if it is shut down or otherwise disconnects from the command queue. If the trigger monitor service is set to restart automatically, the entire configuration becomes very resilient. As an additional benefit, triggering the WebSphere MQ File Transfer Edition agent enables the WebSphere MQ administrator to start and stop it remotely by enabling or disabling triggering on the agent's command queue.

#----------------------------------------------------------------------#
# fteTriggerAgent.pl
#
# Windows program to start a triggered FTE agent
#
# Script parses a WMQ TMC2 tigger message and starts the named FTE agent
# specified in the process definition.  Uses unpack so that the MQSeries
# Perl module is not required.
#
# The command constructed from the process definition is as follows:
# Drive:\path to\fteStartAgent AgentName
#
# The $EnvData contains the fully-qualified executable name which must
# end with fteStartAgent.cmd
#
# The $UserData field must contain the agent name.
#
#
#----------------------------------------------------------------------#
# History
# 20100125 T.Rob - New script
#
#
#----------------------------------------------------------------------#
use strict;
use Fcntl qw(:flock);
use File::Path 'mkpath';
$| = 1; # Turn on Autoflush
($0) = (split(m|[/\\]|, $0))[-1]; # Normalize $0

my $Testing = 1; # Set to 0 to disable dump of trigger messages
my $LogFile;

# Set up the log file name
{
     my ($Min, $Hr, $MDay, $Mon, $Year) = (localtime(time))[1..5];
     $Year = $Year %100; $Mon++;
     my $TimeStamp = sprintf("%02d%02d%02d",$Year,$Mon,$MDay);
     my $LogPath = 'C:\\IBM\\WMQFTE\\Logs';
     my @Created = mkpath( $LogPath );
     $LogFile = "$LogPath\\$0.$TimeStamp.log";
}

# Trigger Message    struct tagMQTMC2 {
 my ($StrucId,      #   MQCHAR4    StrucId;      /* Structure identifier    */
     $Version,      #   MQCHAR4    Version;      /* Structure version number*/
     $QName,        #   MQCHAR48   QName;        /* Name of triggered queue */
     $ProcessName,  #   MQCHAR48   ProcessName;  /* Name of process object  */
     $TriggerData,  #   MQCHAR64   TriggerData;  /* Trigger data            */
     $ApplType,     #   MQCHAR4    ApplType;     /* Application type        */
     $ApplId,       #   MQCHAR256  ApplId;       /* Application identifier  */
     $EnvData,      #   MQCHAR128  EnvData;      /* Environment data        */
     $UserData,     #   MQCHAR128  UserData;     /* User data               */
     $QMgrName,     #   MQCHAR48   QMgrName;     /* Queue manager name      */
     );             # };

&Log("$0 Started", exists($ENV{'COMPUTERNAME'}) ? " on $ENV{'COMPUTERNAME'}" : '');
foreach (@ARGV) {
     if (/^TMC    2/) { # Ignore all parms but TMC2 itself
          ($StrucId, $Version, $QName, $ProcessName, $TriggerData, $ApplType, 
		$ApplId, $EnvData, $UserData, $QMgrName,)
               = unpack("a4 a4 a48 a48 a64 a4 a256 a128 a128 a48", $_);
          $QName    =~ s/\s+$//; # delete trailing spaces
          $ApplId   =~ s/\s+$//; # delete trailing spaces
          $UserData =~ s/\s+$//; # delete trailing spaces
          $EnvData  =~ s/\s+$//; # delete trailing spaces
          $QMgrName =~ s/\s+$//; # delete trailing spaces
      
          &DumpTriggerMsg if $Testing;
      
          # Verify the $EnvData parm
          my $Path;
          if ($EnvData =~ /^(.*)\\bin\\fteStartAgent.cmd$/ && -e $EnvData) {
               $Path = $1;
          } else {
               &Log("$0: Bad command in process ENVRDATA: '$EnvData'");
          }
      
          my @Exec = ($EnvData, $UserData);
          &Log(join(" ", @Exec), "\n");
          system(@Exec);
          last;
     } else {
          if (/^-v/i) {
               $Testing = 1;
               &Log("Verbose mode enabled on command line.");
          } else {
               &Log("Parm not a TMC2 message = '$_'");
          }
     }
}
&Log("$0 Ended.\n\n");
exit 1; # Non-zero exit for trigger monitor

sub DumpTriggerMsg {
     &Log("\$0          = '$0");
     &Log("StrucId      = '$StrucId'");
     &Log("Version      = '$Version'");
     &Log("QName        = '$QName'");
     &Log("ProcessName  = '$ProcessName'");
     &Log("TriggerData  = '$TriggerData'");
     &Log("ApplType     = '$ApplType'");
     &Log("ApplId       = '$ApplId'");
     &Log("EnvData      = '$EnvData'");
     &Log("UserData     = '$UserData'");
     &Log("QMgrName     = '$QMgrName'\n");
}

sub Log {
     my ($Sec, $Min, $Hr, $MDay, $Mon, $Year) = (localtime(time))[0..5];
     my $TimeStamp = sprintf("%04d%02d%02d-%02d:%02d:%02d: ",($Year+1900),
	++$Mon,$MDay,$Hr,$Min,$Sec);

     open LOG, ">>$LogFile";
     # Wait up to 5 seconds for an exclusive lock on log file, else skip logging
     my $Sleep = 5;
     while ($Sleep) {
          # Non-Blocking check for an exclusive lock
          if (flock(LOG, LOCK_EX|LOCK_NB)) {
               print LOG $TimeStamp, @_, "\n";
               $Sleep = 0;
          } else {
               --$Sleep;
          }
     }
     close LOG;
}

Back to top

Download

Information about download methods

Resources

Learn

• WebSphere MQ SupportPacs main page
  
  
• SupportPac MA7K Windows client trigger monitor service
  
  
• SupportPac MQC7 WebSphere MQ V7.0 Clients
  
  
• WebSphere MQ Recommended Fixes (latest fix packs)
  
  
• Podcast: The Deep Queue
  
  
• Author's Web page: T-Rob.net
  
  
• IBM developerWorks WebSphere
  
  

Discuss

• IBMers' Blog on Messaging
  
  
• The Vienna WebSphere MQ List server
  
  
• MQSeries.net
  
  
• developerWorks WebSphere MQ forum
  
  

About the author

T.Rob Wyatt is a Senior Managing Consultant with IBM Software Services for WebSphere who assists customers with the administration, architecture, and security of WebSphere MQ. Recently he has focused on WebSphere MQ security, publishing in the IBM WebSphere Developer Technical Journal and presenting at the IMPACT and European Transaction and Messaging conferences. T.Rob also hosts The Deep Queue, a monthly WebSphere MQ security podcast.

Rate this article

Comments

Back to top

Table of contents

• Introduction
• Set up and test a Windows client trigger monitor
• Conclusion
• Download
• Resources
• About the author
• Comments

Next steps from IBM

• Try: WebSphere MQ File Transfer Edition
• Buy: WebSphere MQ File Transfer Edition

Dig deeper into WebSphere on developerWorks

Try IBM PureSystems. No charge.

Experience today!

---

Get started with the cloud-based trial and pattern development kit.

Special offers