Using an integration node with an existing high availability manager

You can use IBM® App Connect Enterprise with an existing high availability manager, for example HACMP, HA/XD, Veritas Cluster Server (VCS), or HP-UX Serviceguard.

1. To create an integration node, mount the shared resource onto your primary node and use the following mqsicreatebroker command with the -e parameter to specify your shared resource location.

    mqsicreatebroker INODE -e /MQHA/MyIntNode/

   where:

   • INODE is the name of the integration node.
   • /MQHA/INODE/ is the directory for your shared resource.
2. To add another integration node instance, mount the shared resource onto your secondary nodes and use the following mqsiaddbrokerinstance command.

    mqsiaddbrokerinstance INODE -e /MQHA/MyIntNode/

   where:

   • INODE is the name of the integration node.
   • /MQHA/INODE/ is the directory for your shared resource.
3. To start an integration node, you can use one of the following script files:
   hamqsi_start_broker

   #!/bin/ksh
   # Module:
   #   hamqsi_start_broker
   #
   # Args:
   #   BROKER = name of integration node to start
   #
   # Description:
   #   This script attempts to start the MQSI integration node
   #
   #   Runs as the userid which runs integration node, and must have 
   #   the user's environment (i.e. invoke from "su - $MQUSER ..")
   #
   
   BROKER=$1
   
   if [ -z "$BROKER" ]
   then
     echo "hamqsi_start_broker: ERROR! No integration node name supplied"
     echo "   Usage: hamqsi_start_broker <BROKER>" 
     exit 1
   fi
   
   # Ensure that the integration node is not already running. In this test
   # we look for any integration node-related processes, which might have
   # been left around after a previous failure. Any that remain
   # must now be terminated. This is a severe method of cleaning 
   # up integration node processes.
   # We stop the processes in the following order:
   #   bipservice       - first so it cannot issue restarts
   #   bipbroker        - next for same reason
   #   biphttplistener
   #   bipMQTT
   #   startDataFlowEngine
   #  DataFlowEngine  - last
   #
   echo "hamqsi_start_broker: Ensure $BROKER not already running"
   for process in bipservice bipbroker biphttplistener startDataFlowEngine DataFlowEngine
   do
     # Output of kill redirected to /dev/null in case no processes
     ps -ef | grep "$process $BROKER" | grep -v grep | \
        awk '{print $2}'| xargs kill -9 > /dev/null 2>&1
   done
   
   
   # Start the integration node 
   echo "hamqsi_start_broker: Start integration node " $BROKER
   mqsistart $BROKER > /dev/null 2>&1
   if [ $? -ne "0" ]
   then 
     echo "hamqsi_start_broker: Bad result from mqsistart for $BROKER"
     exit 1
   fi
   
   # Check to see if the integration node service has started. This loop 
   # uses a fixed online timeout of approx. 10 seconds.
   TIMED_OUT=yes
   i=0
   while [ $i -lt 10 ]
   do
     # Check for integration node start. We look for bipservice and 
     # bipbroker to be running; there might be no message flows
     # deployed.
     # Look to see whether bipservice is running
     cnt=`ps -ef | grep "bipservice $BROKER" | grep -v grep | wc -l`
     if [ $cnt -gt 0 ]
     then
       # Look to see whether bipbroker is running
       cnt=`ps -ef | grep "bipbroker $BROKER" | grep -v grep | wc -l`
       if [ $cnt -gt 0 ]
       then
         # Integration node is online
         echo "hamqsi_start_broker: ${BROKER} is running"
         TIMED_OUT=no
         break  # out of timing loop
       fi
     fi
     # Manage the loop counter
     i=`expr $i + 1`
     sleep 1
   done
   
   # Report error if integration node failed to start in time
   if [ ${TIMED_OUT} = "yes" ]
   then 
     echo "hamqsi_start_broker: Integration node service failed to start: " $BROKER
     exit 1
   fi
   
   exit 0

   hamqsi_start_broker_as

   #
   #!/bin/ksh
   # Module:
   #   hamqsi_start_broker_as
   #
   # Args:
   #   integration node = integration node name
   #   qm     = name of integration node queue manager
   #   mquser = user account under which QM and Integration node are run
   #
   # Description:
   #   Starting an MQSI integration node requires the following services:
   #   (1) The MQSeries Queue Manager which supports the integration node  
   #   (2) The MQSI integration node service 
   #   This script provides a single source to initiate the required 
   #   services in sequence. 
   #  
   #   Queue Manager:
   #   This script uses the strmqm script supplied by MQSeries V7
   #  
   #   Integration node:
   #   This script then invokes the hamqsi_start_broker script which 
   #   checks that the integration node is fully stopped and then starts it.
   #
   #   The hamqsi_start_broker_as script should be run as root.
   
   
   
   # Check running as root
   if [ `id -u`  -ne 0 ]
   then
     echo "Must be running as root"
     exit 1
   fi
   
    
   BROKER=$1
   QM=$2
   MQUSER=$3
   
   # Check all parameters exist
   
   if [ -z "$BROKER" ]
   then
     echo "hamqsi_start_broker_as: ERROR! No integration node name supplied"
     echo "  Usage: hamqsi_start_broker_as <BROKER> <QM> <MQUSER>"
     exit 1
   fi
   
   if [ -z "$QM" ]
   then
     echo "hamqsi_start_broker_as: ERROR! No queue manager name supplied"
     echo "  Usage: hamqsi_start_broker_as <BROKER> <QM> <MQUSER>"
     exit 1
   fi
   
   if [ -z "$MQUSER" ]
   then
     echo "hamqsi_start_broker_as: ERROR! No Userid supplied"
     echo "  Usage: hamqsi_start_broker_as <BROKER> <QM> <MQUSER>"
     exit 1
   fi
   
   # ------------------------------------------------------------------- 
   # Start the Queue Manager 
   #
   echo "hamqsi_start_broker_as: Start Queue manager " $QM 
   su $MQUSER -c "/opt/mqm/bin/strmqm $QM"
   rc=$?
   if [ $rc -ne 0 ]
   then
     echo "hamqsi_start_broker_as: Could not start the queue manager"
     exit $rc
   fi
   
   # ------------------------------------------------------------------- 
   # Start the Integration node
   #
   # Ensure that the integration node is not already running and start the Integration node 
   su - $MQUSER -c "/MQHA/bin/hamqsi_start_broker $BROKER"
   rc=$?
   if [ $rc -ne 0 ]
   then
     echo "hamqsi_start_broker_as: Could not start the integration node"
     exit $rc
   fi
   
   exit $rc
   

4. To stop an integration node, you can use one of the following script files:
   hamqsi_stop_broker

   #!/bin/ksh
   # Module:
   #   hamqsi_stop_broker
   #
   # Args:
   #   integration node = name of integration node
   #   timeout = max time to allow for each phase of termination
   #
   # Description:
   #   This script stops the integration node, forcibly if necessary.
   #   The script should be run by the user account under which 
   #   the integration node is run, including environment. 
   
   BROKER=$1
   TIMEOUT=$2
   
   if [ -z "$BROKER" ]
   then
     echo "hamqsi_stop_broker: ERROR! No integration node name supplied"
     echo "   Usage: hamqsi_stop_broker <BROKER> <TIMEOUT>"
     exit 1
   fi
   
   if [ -z "$TIMEOUT" ]
   then
     echo "hamqsi_stop_broker: ERROR! No timeout supplied"
     echo "   Usage: hamqsi_stop_broker <BROKER> <TIMEOUT>"
     exit 1
   fi
   
   for severity in normal immediate terminate
   do
     # Issue the stop method in the background - we don't
     # want to risk having it hang us up, indefinitely. We
     # want to be able to concurrently run a TIMEOUT timer
     # to give up on the attempt, and try a more forceful 
     # stop. If the kill version fails then there is nothing 
     # more we can do here anyway.
     
     echo "hamqsi_stop_broker: Attempting ${severity} stop of ${BROKER}"
     case $severity in
   
     normal)
       # Minimum severity of stop is to issue mqsistop
       mqsistop $BROKER > /dev/null 2>&1 &
       ;;
   
     immediate)
       # This is an immediate stop.
       mqsistop $BROKER -i > /dev/null 2>&1 &
       ;;
   
     terminate)
       # This is a severe method of cleaning up integration node processes.
       # We stop the processes in the following order:
       #   bipservice       - first so it cannot issue restarts
       #   bipbroker        - next for same reason
       #   biphttplistener
       #   bipMQTT
       #   startDataFlowEngine
       #   DataFlowEngine  - last
       for process in bipservice bipbroker biphttplistener startDataFlowEngine DataFlowEngine 
       do
         # Output of kill redirected to /dev/null in case no processes
         ps -ef | grep "$process $BROKER" | grep -v grep | \
           awk '{print $2}'| xargs kill -9 > /dev/null 2>&1
       done
       ;;
   
     esac
   
     echo "hamqsi_stop_broker: Waiting for ${severity} stop of ${BROKER} to complete"
     TIMED_OUT=yes
     SECONDS=0
     while (( $SECONDS < ${TIMEOUT} ))
     do
       # See whether there are any integration node processes still running 
       cnt=`ps -ef | \
         grep -E "bipservice $BROKER|bipbroker $BROKER|startDataFlowEngine $BROKER|DataFlowEngine $BROKER|biphttplistener $BROKER" | \
         grep -v grep | wc -l`
       if [ $cnt -gt 0 ]
       then
         # It's still running...wait for timeout
         sleep 1 # loop granularity
       else
         # It's stopped, as desired
         echo "${BROKER} has stopped"
         TIMED_OUT=no
         break # out of while ..offline timeout loop
       fi
     done # timeout loop
   
     if [ ${TIMED_OUT} = "yes" ]
     then
       continue        # to next level of urgency
     else
       break           # instance is stopped, job is done
     fi
   
   done # next level of urgency
   
   if [ ${TIMED_OUT} = "no" ]
   then
     echo "hamqsi_stop_broker: Completed"
     exit 0
   else
     echo "hamqsi_stop_broker: Completed with errors"
     exit 1
   fi 
   

   hamqsi_stop_broker_as

   #!/bin/ksh
   # Module:
   #   hamqsi_stop_broker_as
   #
   # Arguments are:
   #   integration node = name of integration node 
   #   qm     = name of integration node queue manager
   #   mquser = user account under which QM and integration node run
   #   timeout = max time to allow each phase of stop processing
   #
   # Description:
   #   This script stops the integration node, Queue Manager in that sequence.
   #  
   #   Integration node:
   #   The script invokes the hamqsi_stop_broker script to stop the 
   #   integration node, which checks that the integration node is fully stopped.
   #  
   #   Queue Manager:
   #   This script uses the strmqm script supplied by MQSeries V7 
   #  
   #   The hamqsi_stop_broker_as script should be run as root. 
   
   
   # Check running as root
   if [ `id -u`  -ne 0 ]
   then
     echo "Must be running as root"
     exit 1
   fi
   
    
   BROKER=$1
   QM=$2
   MQUSER=$3
   TIMEOUT=$4
   
   # Check all parameters
   
   if [ -z "$BROKER" ]
   then
     echo "hamqsi_stop_broker_as: ERROR! No Integration node name supplied"
     echo "   Usage: hamqsi_stop_broker_as <BROKER> <QM> <MQUSER> <TIMEOUT>"
     exit 1
   fi
   
   if [ -z "$QM" ]
   then
     echo "hamqsi_stop_broker_as: ERROR! No queue manager name supplied"
     echo "   Usage: hamqsi_stop_broker_as <BROKER> <QM> <MQUSER> <TIMEOUT>"
     exit 1
   fi
   
   if [ -z "$MQUSER" ]
   then
     echo "hamqsi_stop_broker_as: ERROR! No userid supplied"
     echo "   Usage: hamqsi_stop_broker_as <BROKER> <QM> <MQUSER> <TIMEOUT>"
     exit 1
   fi
   
   if [ -z "$TIMEOUT" ]
   then
     echo "hamsi_stop_broker_as: ERROR! No Timeout value supplied"
     echo "   Usage: hamqsi_stop_broker_as <BROKER> <QM> <MQUSER> <TIMEOUT>"
    exit 1
   fi
   
   
   METHOD_STATUS="OK"
   
   # ------------------------------------------------------------------- 
   # Stop the integration node
   #
   echo "hamqsi_stop_broker_as: Stop integration node " $BROKER
   su - $MQUSER -c "/MQHA/bin/hamqsi_stop_broker $BROKER $TIMEOUT"
   if [ $? -ne "0" ]
   then
     # Even if the above operation failed, just report and then continue by 
     # stopping other components
     echo "hamqsi_stop_broker_as: Attempt to stop integration node $BROKER failed"
     METHOD_STATUS="Error"
   fi
   
   # ------------------------------------------------------------------- 
   # Stop the Queue Manager, using script from MQ V7
   #
   echo "hamqsi_stop_broker_as: Stop Queue Manager $QM"
   su $MQUSER -c "/opt/mqm/bin/endmqm -i $QM"
   if [ $? -ne "0" ]
   then
     # Even if the above operation failed, just report and then continue by 
     # stopping other components
     echo "hamqsi_stop_broker_as: Attempt to stop queue manager $QM failed"
     METHOD_STATUS="Error"
   fi
   
   if [ ${METHOD_STATUS} = "OK" ]
   then
     exit 0
   else
     echo "hamqsi_stop_broker_as: Completed with errors"
     exit 1
   fi

5. To monitor an integration node, you can use the following script file that checks only for the existence of the main integration node processes and provides a successful return code if they are found:
   hamqsi_monitor_broker_as

   #!/bin/ksh
   # Module:
   #   hamqsi_monitor_broker_as
   #
   # Args:
   #   BROKER = name of integration node in AppServer
   #   QM     = name of queue manager in AppServer
   #   MQUSER = userid under which queue manager and integration node run
   # 
   # Description:
   #   This is the application monitor script used with HACMP/ES. It 
   #   needs to be invoked by a parameter-less wrapper script because
   #   HACMP does not allow parameters to be passed to application
   #   monitor scripts.
   #  
   #   This hamqsi_monitor_broker_as script is run as root, and uses 
   #   su as needed to monitor the 3 components of the application server.
   #  
   #   This script is tolerant of a queue manager that is still in
   #   startup. If the queue manager is still starting this application 
   #   monitor script will exit with 0 - which indicates
   #   to HACMP that there's nothing wrong. This is to allow for
   #   startup time for the queue manager which might exceed the 
   #   Stabilisation Interval set for the Application Monitor in HACMP/ES.
   #    
   #  
   # Exit codes:
   #   0  => Integration node & QM are all running OK or starting
   #   >0 => One or more components are not responding.
   #
   
   # Check running as root
   if [ `id -u`  -ne 0 ]
   then
     echo "Must be running as root"
     exit 1
   fi
   
   BROKER=$1
   QM=$2
   MQUSER=$3
   
   # Check the parameters
   
   if [ -z "$BROKER" ]
   then
     echo "hamqsi_monitor_broker_as: ERROR! No integration node name supplied"
     exit 1
   fi
   
   if [ -z "$QM" ]
   then
     echo "hamqsi_monitor_broker_as: ERROR! No queue manager name supplied"
     exit 1
   fi
   
   if [ -z "$MQUSER" ]
   then
     echo "hamqsi_monitor_broker_as: ERROR! No mquser supplied"
     exit 1
   fi
   
   # Use a state variable to reflect the state of components as they
   # are tested. Valid values are "stopped", "starting" and "started"
   # Initialise it to "stopped" for safety. 
   STATE="stopped"
   
   # ------------------------------------------------------------------
   # Check that the queue manager is running or starting.
   #
   su - $MQUSER -c "echo 'ping qmgr' | runmqsc ${QM}" > /dev/null 2>&1
   pingresult=$?
   # pingresult will be 0 on success; non-zero on error (man runmqsc)
   if [ $pingresult -eq 0 ]
   then 
     # ping succeeded
     echo "hamqsi_monitor_broker_as: Queue Manager ${QM} is responsive"
     STATE="started"
   else 
     # ping failed
     # Don't condemn the QM immediately, it might be in startup.
     # The following regexp includes a space and a tab, so use tab-friendly
     # editors.
     srchstr=" $QM[  ]*$"
     cnt=`ps -ef | grep strmqm | grep "$srchstr" | grep -v grep \
                   | awk '{print $2}' | wc -l`
     if [ $cnt -gt 0 ]
     then
       # It appears that QM is still starting up, tolerate
       echo "hamqsi_monitor_broker_as: Queue Manager ${QM} is starting"
       STATE="starting"
     else
       # There is no sign of QM start process
       echo "hamqsi_monitor_broker_as: Queue Manager ${QM} is not responsive"
       STATE="stopped"
     fi
   fi
   
   
   # Decide whether to continue or to exit
   case $STATE in
     stopped)
       echo "hamqsi_monitor_broker_as: Queue manager ($QM) is not running correctly"
       exit 1  
       ;;
     starting)
       echo "hamqsi_monitor_broker_as: Queue manager ($QM) is starting"
       echo "hamqsi_monitor_broker_as: WARNING - Stabilisation Interval might be too short"
       echo "hamqsi_monitor_broker_as: WARNING - No test of integration node $BROKER will be conducted"
       exit 0  
       ;;
     started)
       echo "hamqsi_monitor_broker_as: Queue manager ($QM) is running"
       continue 
       ;;
   esac
   
   # ------------------------------------------------------------------
   # Check the MQSI integration node is running 
   # 
   # Re-initialise STATE for safety
   STATE="stopped"
   #
   # The integration node runs as a process called bipservice which is responsible
   # for starting and re-starting the admin agent process (bipbroker). 
   # The bipbroker is responsible for starting any DataFlowEngines. The 
   # bipbroker starts the DataFlowEngines using the wrapper script 
   # startDataFlowEngine. If no integration servers have been assigned to 
   # the integration node there will be no DataFlowEngine processes. There should 
   # always be a bipservice and bipbroker process pair. This monitor 
   # script only tests for bipservice, because bipservice should restart 
   # bipbroker if necessary - the monitor script should not attempt to 
   # restart bipbroker and it might be premature to report an absence 
   # of a bipbroker as a failure.
   #
   cnt=`ps -ef | grep "bipservice $BROKER" | grep -v grep | wc -l`
   if [ $cnt -eq 0 ]
   then 
     echo "hamqsi_monitor_broker_as: MQSI integration node $BROKER is not running"
     STATE="stopped"
   else
     echo "hamqsi_monitor_broker_as: MQSI integration node $BROKER is running" 
     STATE="started"
   fi
   
   # Decide how to exit
   case $STATE in
     stopped)
       echo "hamqsi_monitor_broker_as: Integration node ($BROKER) is not running correctly"
       exit 1  
       ;;
     started)
       echo "hamqsi_monitor_broker_as: Integration node ($BROKER) is running"
       exit 0 
      ;;
   esac 

   If you require more information than that supplied by the preceding example, you can code your monitor to do the following actions:

   • Subscribe to IBM App Connect Enterprise accounting and statistics, and analyze the results.
   • Put a dummy message through the integration node and analyze the results.
6. Before you delete the integration node on the primary node, you must delete any integration nodes on the standby nodes. For more information, see Deleting an integration node.