Using Python in WebSphere MQ applications

PyMQI, an open-source Python extension for WebSphere MQ, is an open-source library built on top of the WebSphere MQ MQI library. This article shows you how PyMQI lets you use the power of Python to developing tools and applications for WebSphere MQ.

Sami Salkosuo (sami.salkosuo@fi.ibm.com), Application Architect, EMC

Sami Salkosuo is a Software Architect with IBM Global Business Services in Helsinki. Has worked at IBM for eight years, and his main area of interest is programming in Java, Python, and PHP. His certifications include Sun Certified Java Programmer, IBM Certified Solution Developer for XML and Related Technologies, and IBM Certified Solution Designer for WebSphere MQ. You can contact Sami at sami.salkosuo@fi.ibm.com.



29 August 2007

Also available in

Python MQ Interface

Python MQ Interface (PyMQI) is an open-source Python extension library for IBM® WebSphere® MQ. PyMQI extends the MQI library by providing an easy-to-use Python interface. , It includes a low-level library written in C to access the MQI C-library, and a high-level object-oriented interface, on top of low-level library, for Python programmers. PyMQI lets developers use the powerful Python language in WebSphere MQ tools and solutions, thus giving the benefits of Python to all WebSphere MQ developers. A quote from the Python homepage makes this point:

"Python is a dynamic object-oriented programming language... that comes with extensive standard libraries and can be learned in a few days... Programmers report substantial productivity gains... The language encourages the development of higher quality, more maintainable code."

Compiling PyMQI

PyMQI is distributed as source, so the first thing you need to do after downloading and extracting the package to a directory is to compile it. Typically, Python extensions are compiled with the same compiler as Python itself. Python distributions are built with Microsoft® Visual Studio, and if you have the correct version installed you can build PyMQI using the command python setup.py build server (for WebSphere MQ Server V5) or python setup.py build client (for WebSphere MQ Client V5)

If you do not have Microsoft Visual Studio and you are using WebSphere MQ V6, compiling requires additional steps before you can use PyMQI:

  1. Install MinGW32, minimalist GNU for the Windows® C-compiler.
  2. Modify WebSphere MQ C-header files.
  3. Compile PyMQI.
  4. Install PyMQI as a Python extension.

The compile requires that Windows, Python V2.5.1, and WebSphere MQ V5 or V6 Server or Client is installed in your computer. For this article, the environment included Windows XP 2003, WebSphere MQ V6 Server, and Python V2.5.1.

Install MinGW32

Download MinGW-5.1.3.exe and execute it to start installation. The installation program downloads required packages from the Web and installs them. A full install installs all compilers, though a minimum install may be sufficient. After installing MinGW, add the MinGW\bin-directory to your PATH environment variable.

Modify WebSphere MQ C-header files

If you are using WebSphere MQ V6, you need to make a couple of small changes to the MQ C-header files, because MinGW does not understand a couple of 64-bit type definitions.

Open the header file <MQ_INSTALL_DIR>\Tools\c\include\cmgc.h in a text editor and find the code in Listing 1:

Listing 1. Typedefs to be replaced
typedef _int64 MQINT64;
typedef unsigned _int64 MQUINT64;

Delete or comment out the typedefs in Listing 1. Then add the new typedefs shown in Listing 2 and save the file.

Listing 2. Typedefs to be replaced
typedef long long _int64;
typedef unsigned long long _uint64;
typedef _int64 MQINT64;
typedef _uint64 MQUINT64;

Compile PyMQI

Open a command prompt in the directory where you extracted the PyMQI source files and compile PyMQI using the command setup.py build -cmingw32 server. This command compiles the PyMQI extensions to prepare them for installation as Python extensions.

Install PyMQI as a Python extension

To install PyMQI as a Python extension, use the command setup.py install --skip-build. PyMQI is now installed and ready to be used.

Programming with PyMQI

PyMQI has several modules that are used together:

  • CMQC and CMQCFC define the MQI constants.
  • pymqe is a low-level Python extension interface to MQI, written in C.
  • pymqi is a high-level Python interface to MQ that uses pymqe.

In practice, pymqi is used when developing WebSphere MQ applications. Listing 3 shows the minimum code to put a message in a queue:

Listing 3. Put message to queue
import pymqi 
qmgr = pymqi.QueueManager()
putQ = pymqi.Queue(qmgr, 'QNAME')
putQ.put('Hello world!')

Getting a message from a queue is also simple:

Listing 4. Get message from queue
import pymqi 
qmgr = pymqi.QueueManager()
getQ = pymqi.Queue(qmgr, 'QNAME')
print 'Message:', getQ.get()

PyMQI supports major MQI calls such as MQCONN/MQCONNX, MQDISC, MQOPEN, MQCLOSE, MQPUT, MQGET, and others. PCF commands are also supported as well as transactions.

The next section includes some useful utilities written with PyMQI: PyPut, PyGet and PyBrowse. A WebSphere MQ queue manager named TEST_QM and a local queue named TESTQ are created before executing any of the programs.

PyMQI utilities

This section has examples of PyMQI and presents useful tools for WebSphere MQ developers and administrators. PyPut is a tool to put one or more messages (specified as argument or a file) to a queue. PyGet gets messages from a queue and prints them to system out or saves them to a file either one message at a time, or all messages at once. PyBrowse browses messages -- it can also browse a specific message in a queue or browse messages in last-in-first-out (LIFO) order instead of the normal first-in-first-out (FIFO) order.

You can download the sample code and use it as-is -- just copy and paste the code to a file and execute it.

PyPut

Listing 5 has the code for PyPut. in addition to pymqi, PyPut uses Python module OptionParser for defining command line options:

Listing 5. PyPut.py
from pymqi import *
import sys, os
from optparse import OptionParser

usage='Usage: %s [options] QMGRNAME QNAME (MSSG | FILE_NAME) [(MSG | FILE_NAME)...]'

def main(argv=None):
    if argv is None:
      argv = sys.argv
    try:
      (options,args)=parseOptions()
      if len(args)<3:
        raise MQException(1)
      qmgrName=args[0]
      qName=args[1]
      message=args[2]
      messages=args[2:]
      #open queue in specified queue manager
      qmgr = QueueManager(qmgrName)
      putQ = Queue(qmgr, qName)
      #loop through arguments and put messages
      #to a queue
      for message in messages:
        #if -f options is specified, arguments are file names
        if options.msgIsFile==True:
          if os.path.exists(message)==False:
            raise MQException(3)
          #read file contents to a message-variable
          f=open(message)
          content=f.readlines()
          f.close()
          #variable 'content' is a list of lines in the specified file
          #join lines in content-variable together as one string
          message=''.join(content)
        
        #single method call to put message to a queue
        putQ.put(message)
        if not options.quiet==True:
          print 'Message sent.' 
    except MQException, err:
      #if exception happens, print error message and exit
      print  '[ERROR]',err.errorCodes[err.msg]
      print usage % (sys.argv[0],)
      return err.code
    except MQMIError,err:
      print err
      return -1

def parseOptions():  
  parser = OptionParser(usage=(usage % ('%prog',)))
  parser.add_option("-f", 
                    "--file", 
                    action="store_true",
                    dest="msgIsFile",
                    help="Use file specified as third argument as message .", )
  parser.add_option("-q", 
                    "--quiet", 
                    action="store_true",
                    dest="quiet",
                    help="Quiet mode.", )
  (options, args) = parser.parse_args()
  return (options, args)


#MQException class that holds application specific error codes
#and error messages
class MQException(Exception):
  errorCodes={'1': 'QMGRNAME, QNAME or MESSAGE missing.',
              '2': 'Too many arguments.',
              '3': 'File does not exist.',
              '4': 'QMGRNAME or QNAME.',
              }
  code=0
  def __init__(self, code):
    self.code=code
    self.msg = str(code)

if __name__ == "__main__":
    sys.exit(main())

The code in Listing 5 above is mainly application logic and the actual PyMQI code is only a few lines: get queue manager, get queue, put message to a queue, and error handling. This shows the power of PyMQI: using WebSphere MQ in Python programs is trivial and built-in features of Python makes developing tools very easy. For example, reading file contents is four lines of code and it could be a function in your own toolbox. There are -h and -help options in the Python OptionParser-module to print the command-line help. For example PyPut.py -h has help for PyPut:

Usage: PyPut.py [options] QMGRNAME QNAME (MSSG | FILE_NAME) [(MSG | FILE_NAME)...]

Options:
  -h, --help   show this help message and exit
  -f, --file   Use file specified as third argument as message .
  -q, --quiet  Quiet mode.

Execute PyPut using the command PyPut.py TEST_QM TESTQ message1 message2. This command puts two messages to a queue named TESTQ in queue manager TEST_QM.

PyPut.py can be also used in scripts. For example: @for %i in ("*.txt") do call "PyPut.py" "-f" "TEST_QM" "TESTQ" "%i" loops all txt files in the current directory and puts the contents of each txt file to a queue.

PyGet

PyGet.py is similar to PyPut.py except that messages are printed to system out or written to a file. Listing 6 shows the code for PyGet.py:

Listing 6. PyGet.py
from pymqi import *
import sys, os
from optparse import OptionParser

usage='Usage: %s [options] QMGRNAME QNAME'
def main(argv=None):
    if argv is None:
      argv = sys.argv
    try:
      (options,args)=parseOptions()
      if len(args)>2:
        raise MQException(2)
      if len(args)<2:
        raise MQException(4)
      qmgrName=args[0]
      qName=args[1]
      
      #open queue manager and queue
      qmgr = QueueManager(qmgrName)
      Q = Queue(qmgr, qName)      
      if options.all==True:
        moreMessages=True
        while(moreMessages):
          try:
            #get single message from queue
            message=Q.get()
            printMessage(options,message)
          except MQMIError,err:
            if err.reason==2033:
              moreMessages=False
              if not options.quiet==True:
                if not options.noDelim==True:
                  print '-'*70
                print 'No messages.' 
            else:
              raise
          
      else:
        message=Q.get()
        printMessage(options,message)
    except MQException, err:
      print  '[ERROR]',err.errorCodes[err.msg]
      print usage % (sys.argv[0],)
      return err.code
    except MQMIError,err:
      if err.reason==2033:
        #no messages
        if not options.quiet==True:
          print 'No messages.' 
          
      else:
        print err
        return err.reason

def printMessage(options,message):
  if options.saveFile==True:
    seq=1
    if not options.fileName==None:
      fileName=options.fileName
    else:
      fileName='message'
      fileName='%s%05d' %(fileName,seq)
    while os.path.exists(fileName)==True:
      fileName=fileName[:-5]
      seq=seq+1
      fileName='%s%05d' %(fileName,seq)
    f=open(fileName,'w')
    f.write(message)
    f.close()
  else:
    if not options.quiet==True:
      if not options.noDelim==True:
        print '-'*70
      print message

def parseOptions():  
  parser = OptionParser(usage=(usage % ('%prog',)))
  parser.add_option("-f", 
                    "--file", 
                    action="store_true",
                    dest="saveFile",
                    help="Save message to a file with name 'message'.", )
  parser.add_option("-n", 
                    "--file-name", 
                    dest="fileName",
                    help="File name for message. Use with -f option.", )
  parser.add_option("-A", 
                    "--all", 
                    action="store_true",
                    dest="all",
                    help="Get all messages. If messages are saved to file,"+
                     " messages have sequence number in the file name.", )
  parser.add_option("-q", 
                    "--quiet", 
                    action="store_true",
                    dest="quiet",
                    help="Quiet mode.", )
  parser.add_option("", 
                    "--no-delim", 
                    action="store_true",
                    dest="noDelim",
                    help="Do not print message delimeter.", )
  
  (options, args) = parser.parse_args()
  return (options, args)

#MQException class that holds application specific error codes
#and error messages
class MQException(Exception):
  errorCodes={'1': 'QMGRNAME, QNAME or MESSAGE missing.',
              '2': 'Too many arguments.',
              '3': 'File does not exist.',
              '4': 'QMGRNAME or QNAME missing.',
              }
  code=0
  def __init__(self, code):
    self.code=code
    self.msg = str(code)
    
if __name__ == "__main__":
    sys.exit(main())

As in PyPut.py, the actual WebSphere MQ specific code is only a few lines and the application logic takes most of the source code. To get help for PyGet, type PyPut.py -h:

Usage: PyGet.py [options] QMGRNAME QNAME

Options:
  -h, --help            show this help message and exit
  -f, --file            Save message to a file with name 'message'.
  -n FILENAME, --file-name=FILENAME
                        File name for message. Use with -f option.
  -A, --all             Get all messages. If messages are saved to file,
                        messages have sequence number in the file name.
  -q, --quiet           Quiet mode.
  --no-delim            Do not print message delimeter.

Several options can be used with PyGet.py. Without options, it retrieves one message from the queue and prints it to system out. With the option -A, all messages are retrieved from the queue, which is useful for clearing the queue, which and you may need to do frequently during development. For example: PyGet.py -A -q TEST_QM TESTQ clears TESTQ without printing anything to system out. When this command is included in a script, a single command clears queue.

PyBrowse

PyBrowse.py has similar features as in PyGet.py, but it browses messages and does not have quiet mode. Additional options are --index, --lifo, --depth. The index option browses a message in a specified index -- index 0 is the first message, index 1 is the second message and so on. The lifo option reverses the messages in the queue before printing them to system out, and can be used to browse the last message put in the queue. The depth option returns the current depth of the queue using the inquery function.

Source code of PyBrowse:

Listing 7. PyBrowse.py
from pymqi import *
import sys, os
from optparse import OptionParser

usage='Usage: %s [options] QMGRNAME QNAME'
def main(argv=None):
    if argv is None:
      argv = sys.argv
    try:
      (options,args)=parseOptions()
      if len(args)>2:
        raise MQException(2)
      if len(args)<2:
        raise MQException(4)
      qmgrName=args[0]
      qName=args[1]
      
      #open queue manager 
      qmgr = QueueManager(qmgrName)
      gqdesc = od( ObjectName=qName)
      Q= Queue( qmgr, gqdesc)
      qDepth=Q.inquire(CMQC.MQIA_CURRENT_Q_DEPTH)
      if options.depth==True:
        #open queue for inquire
        print 'Current depth: %d' % qDepth
        return 0

      if not options.index==None:
        if int(options.index)>=qDepth:
          raise MQException(5)

      #open queue for browsing
      Q= Queue( qmgr, gqdesc, CMQC.MQOO_BROWSE)
      msgDesc = md()
      getOpts = gmo(Options = CMQC.MQGMO_BROWSE_NEXT)
      getOpts.WaitInterval = CMQC.MQWI_UNLIMITED

      messages = []
      try:
        while 1:
          msg = Q.get(None, msgDesc, getOpts)
          messages.append(msg)
          if not options.all==True and options.index==None and not options.lifo==True:
            break

          # null MsgId and CorrelId, or cursor won't move up
          # the Q
          msgDesc['MsgId'] = ''
          msgDesc['CorrelId'] = ''
      except MQMIError, me:
        #assume that error is raised when there are
        #no more messages in queue
        pass
      
      if options.lifo==True:
        messages.reverse()
      
      if not options.all==True and options.index==None:
        messages=messages[0:1]
      
      if not options.index==None:
        printMessage(options,messages[int(options.index)])
      else:
        for message in messages:
          printMessage(options,message)
        
    except MQException, err:
      print  '[ERROR]',err.errorCodes[err.msg]
      print usage % (sys.argv[0],)
      return err.code
    except MQMIError,err:
      if err.reason==2033:
        #no messages
        print 'No messages.' 
      else:
        print err
        return err.reason


def printMessage(options,message):
  if options.saveFile==True:
    seq=1
    if not options.fileName==None:
      fileName=options.fileName
    else:
      fileName='message'
      fileName='%s%05d' %(fileName,seq)
    while os.path.exists(fileName)==True:
      fileName=fileName[:-5]
      seq=seq+1
      fileName='%s%05d' %(fileName,seq)
    f=open(fileName,'w')
    f.write(message)
    f.close()
  else:
    if not options.noDelim==True:
      print '-'*70
    print message

def parseOptions():  
  parser = OptionParser(usage=(usage % ('%prog',)))
  parser.add_option("-f", 
                    "--file", 
                    action="store_true",
                    dest="saveFile",
                    help="Save message to a file with name 'message'.", )
  parser.add_option("-n", 
                    "--file-name", 
                    dest="fileName",
                    help="File name for message. Use with -f option.", )
  parser.add_option("-i", 
                    "--index", 
                    dest="index",
                    help="Browse message in the middle of queue.", )
  parser.add_option("-A", 
                    "--all", 
                    action="store_true",
                    dest="all",
                    help="Browse all messages. If messages are saved to file,"+
                     " messages have sequence number in the file name.", )
  parser.add_option("-d", 
                    "--depth", 
                    action="store_true",
                    dest="depth",
                    help="Print just the queue depth.", )
  parser.add_option("-l", 
                    "--lifo", 
                    action="store_true",
                    dest="lifo",
                    help="Print last message first (Last-In-First-Out).", )
  parser.add_option("", 
                    "--no-delim", 
                    action="store_true",
                    dest="noDelim",
                    help="Do not print message delimeter.", )
  
  (options, args) = parser.parse_args()
  return (options, args)

#MQException class that holds application specific error codes
#and error messages
class MQException(Exception):
  errorCodes={'1': 'QMGRNAME, QNAME or MESSAGE missing.',
              '2': 'Too many arguments.',
              '3': 'File does not exist.',
              '4': 'QMGRNAME or QNAME missing.',
              '5': 'Index error. Not so many messages in queue.',
              }
  code=0
  def __init__(self, code):
    self.code=code
    self.msg = str(code)
    
if __name__ == "__main__":
    sys.exit(main())

Help for PyBrowse:

Usage: PyBrowse.py [options] QMGRNAME QNAME

Options:
  -h, --help            show this help message and exit
  -f, --file            Save message to a file with name 'message'.
  -n FILENAME, --file-name=FILENAME
                        File name for message. Use with -f option.
  -i INDEX, --index=INDEX
                        Browse message in the middle of queue.
  -A, --all             Browse all messages. If messages are saved to file,
                        messages have sequence number in the file name.
  -d, --depth           Print just the queue depth.
  -l, --lifo            Print last message first (Last-In-First-Out).
  --no-delim            Do not print message delimeter.

Building executables

In some cases, users of your tools will not have Python installed and therefore it is useful to have executables for distribution. Fortunately, using Python and the py2exe extension makes it easy to convert Python scripts into Windows executables and py2exe includes all required Python files in a single directory that can be distributed to users. To create executables of your Python scripts, first install py2exe, then create a setup script. Listing 8 has minimal setup script for Python MQ tools:

Listing 8. setup.py
from distutils.core import setup
import py2exe

setup(
    console = ["PyPut.py"],
    )

setup(
    console = ["PyGet.py"],
    )

setup(
    console = ["PyBrowse.py"],
    )

Execute setup.py in the same directory where Python MQ scripts are located. Use the command python setup.py py2exe and wait until it is finished. You now have a dist directory that includes Python MQ scripts as executables and all the required files for executing PyMQ tools. If you look in the dist directory, you see that py2exe included all of the required WebSphere MQ dll files. To distribute your scripts to users who do not have WebSphere MQ installed (for example, if your tools are remote admin tools) you can leave them there -- otherwise you should delete all dll files except python25.dll from the dist directory. Scripts will find the MQ dll files from the system path.

Summary

With the Python MQ Interface, you can use the Python language to develop utilities and programs for WebSphere MQ. Because Python is a powerful tool for rapid development, PyMQI extends rapid development to WebSphere MQ programs. Solutions can be quickly tested and verified using Python before moving to production. The tools described in this article are samples that can be used in everyday work when developing solutions that use WebSphere MQ. Because of Python and PyMQI, sample code will evolve over time and the samples can become assets in your developer or administrator toolbox.


Download

DescriptionNameSize
Sample codePyMQTools.zip5 KB

Resources

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere
ArticleID=252499
ArticleTitle=Using Python in WebSphere MQ applications
publish-date=08292007