Really Small Message Broker
A broker based on MQTT protocol which takes roughly up only 50KB of storage space and can run in only 150KB or less of memory, hence a perfect suite for embedded devices/low-powered servers.
Amazingly it has bridging capabilities to WebSphere Message Broker as well as Lotus Expeditor micro broker also !
Please have a look at RSMB
Blog by
Vijay Raj K and Neeraj Krishna
Redbook on Lotus Expeditor micro broker
A new red book for micro broker version 3 is out there.
Here is the link Messaging from the Edge by Using IBM Lotus Expeditor Client V6.2
This red book basically covers the new features like SSL support and MQTT version 5 client. This also has an enormous collection of design patterns that can be applied to the edge network which helps users/architects to design and deploy their solutions.
Blog by,
Vijay Raj K and Neeraj Krishna
WMB: Procedure to setup, configure and run SAP Connectivity Message Broker sample.
This blog describes in detail how to configure the SAP Server for running the SAP connectivity sample provided with WebSphere Message Broker (WMB). The blog further details the procedure to follow to configure SAP System for SAP Inbound Sample and configuration of SAP Inbound Adapter on Message Broker end.
- Install WebSphere Message Broker 6.1 and upgrade to latest fixpac 03 (6.1.0.3) so that no errors are reported while importing the SAP Connectivity sample.
- Users who wish to run SAP Connectivity sample need to have a login to SAP server and to do this they need to install SAPGUI to login to SAP server.
The SAPGUI that needs to be installed is "SAP - SAPGUI 7.10+Add-ons with SAPLOGON Customizer"
- Once SAPGUI is installed open the connection and add the login details as follows:
Click on SAP LOGON select Systems tab and click on ‘new item’ then press ‘Next’ to go to the following screen and fill in the SAP server details
Save this setting and now we are ready to logon to the SAP Server.
For successful deployment of message flows with SAP nodes, we must configure the message broker runtime with the prerequisite
SAP Java Connector libraries (SAP JCO) and JAR file must be made available to the Adapter Connection Project.The JCo libraries and JAR files is available at the following location
Users need to have valid user id and password to access the above files.
Unzip the files under C:\SAP_JCO
.
- Configuring SAP system (for SAP Inbound sample)
We need to configure the SAP Server to identify the broker as target destination.
Create the RFC connection from the SAP system to the broker
- Log in to your SAP system.
- Go to the SM59 transaction, Configuration of RFC connections.
- Click TCP/IP connections and click Create.
- Check that the connection type is set to T
- Set RFC Destination name to an appropriate value, for example SAMPRFC1.
- Select Registered Server Program in the Technical Settings tab
- Set the Program ID to match the Program ID that is specified in the broker adapter, for example, SAMPRFC1.
- Click Save to save the settings.
- Enter a description in the Description 1 field, for example, RFC for Sample.
- Enter a description in the Description 2 field, for example, your name.
- Click MDMP & Unicode.
- Select Unicode.
- Click the green tick on the pop-up window.
- Click Save
Read the full blog here (Please download and rename the SAP_CONNECTIVITY_DOC.zip.jpg to SAP_CONNECTIVITY_DOC.zip and then unzip)
( Dec 19 2008, 12:45:49 AM EST ) Permalink Comments [0]Redbook : WebSphere MQ V7.0 Features and Enhancements
WMB: Parsing messages with Message domain
This first section of blog explains how to parse messages which contains Message Domain values set in the RFH2 header and the second section explains how to parse the length encoded data basically created by the TCP/IP clients or server using WMB.
Section 1
We may come across with processing some messages created by an adaptor with having MRM domain set in the RFH2 header also set.
The constraint is that we can’t do any changes in the message created by the adaptor. Still we are required to process the message through the message broker using MQInput Node.
Here the message contains wrong Msg Set name as “4” in the RFH2 header and we are not suppose to change the message set name and still process it successfully through the MRM parser used message broker flow.
How to resolve this problem?
Create the following flow using MQInput Node àResetContentDescriptor NodeàCompute Node à MQOutput Node.
Set the following properties in the MQInput Node and ResetContentDescriptor Nodes respectively
MQInput Node doesn’t override or reset the Message Domain properties as mentioned in the above property editor. Since the message set name and message format are not correct in the input message coming from an adaptor, we need to reset them using ResetContentDescriptor node as mentioned below. Here the Message set BIB_ICBS_Messages and CwXML message format is created in the WMB Tookit to parse the Adaptor messages.
The compute Node “ProcessMessage” can be used to manipulate the incoming message. Its optional depends on the business requirements.
Section 2
Parsing length encoded data:
In a banking domain, sometimes the message comes from TCPIP client or TCPIP server, with having data length and data embedded in the message itself.
For example:
Msg1:
00550800822000000000000004000000000000000729110323203140001
Msg2:
0069020022200000080200003000000502114530023105050211023105012DANPACIN2008
In the above message the first four characters shows the length of the message and after that everything is data.
How to split the above messages into length and data using a MRM parser?
You can select data element separation as “Use data pattern” here.
For “length” field type: Set Data Pattern as “.{4}”
Similarly,
For “data” field type: Set Data Pattern as “[A-Za-z0-9]*” and Specify the Length Reference as “length”.
Blog By,
Abhinav Priyadarshi (pabhinav-AT-in.ibm-DOT-com)
( Nov 02 2008, 01:09:53 AM EST ) Permalink Comments [0]WebSphere MQ v6.0.2.5 is now available for iSeries
WebSphere MQ v6.0.2.5 is now available for iSeries.
Please see the Recommended Fixes for Websphere MQ website -> http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037
#1 for more information.( Oct 30 2008, 11:27:17 PM EDT ) Permalink Comments [0]
Installation and uninstallation of WebSphere MQ on Linux
This article outlines the sequence in which the packages have to be installed / uninstalled on Linux. The dependencies of the installation packages are also explained.
Installation
Following is the sequence in which various WebSphere MQ v6.0 for Linux packages have to be installed.
- Gskit
- MQSeriesRuntime
- MQSeriesJava
- MQSeriesSDK
- MQSeriesKeyMan
- MQSeriesServer
- MQSeriesClient
- MQSeriesSamples
- MQSeriesTXClient
- MQSeriesIES30
- MQSeriesConfig
- MQSeriesFTA
- MQSeriesMsg_en
Uninstallation
The following is the sequence in which packages must be uninstalled. This sequence is essentially the reverse of the sequence of installation.
- MQSeriesMsg_en
- MQSeriesFTA
- MQSeriesConfig
- MQSeriesIES30
- MQSeriesTXClient
- MQSeriesSamples
- MQSeriesClient
- MQSeriesServer
- MQSeriesKeyMan
- MQSeriesSDK
- MQSeriesJava
- MQSeriesRuntime
Dependencies
|
Package Name |
Install dependencies |
|
MQSeriesRuntime |
No dependency |
|
MQSeriesJava |
MQSeriesRuntime |
|
MQSeriesSDK |
MQSeriesRuntime |
|
MQSeriesKeyMan |
Gskit |
|
MQSeriesServer |
MQSeriesRuntime. |
|
MQSeriesClient |
MQSeriesRuntime. |
|
MQSeriesSamples |
No dependency |
|
MQSeriesTXClient |
MQSeriesClient MQSeriesRuntime MQSeriesJava |
|
MQSeriesIES30 |
No Dependency |
|
MQSeriesConfig |
MQSeriesRuntime MQSeriesIES30 MQSeriesJava |
|
MQSeriesFTA |
MQSeriesIES30 |
|
MQSeriesMsg_en |
MQSeriesRuntime |
Follow the above sequence and use '-ihv' for installation and '-e' flag for uninstallation.
Note : For abnormalities --nodeps, --noscripts and --replacefiles are used along with rpm. The gskit can be updated specifying -U option.
Lakshmi B Gangigatte and Abhinav Priyadarshi
India Software Labs
Bangalore
Tools for diagnosing Java related issues
As some parts of the messaging software use java, we feel its important to give a summary of java tools that help in diagnosing some runtime issues. Micro broker is 100% java, WebSphere MQ client API is developed in java and there are parts of Message broker that is developed in java. There are situations when the messaging client or the server runs out of memory or start running very slow or gets into a deadlock. The problem might lay in the software that is developed on top of the messaging layer.The first step is install the IBM Support Assistant. The support assistant tool is developed on top of expeditor platform and provides the user the ability to download and install the required tools. Its got the ability to update the tools also easily, hence enables you to worry less to managing these tools.
To get all the java related tools. Just go to Updates -> Find New -> Tools Add-on. You'll now be presented with a list of diagnostic tools for java under "JVM based tools". Its also good to download the "Tech Preview" tools also.
After installing them you can launch them using "Launch Activity" -> Analyze Problem -> Tools tab and choose a tool.
Here is a summary of tools:
- IBM Thread and Monitor Dump Analyzer - For analyzing dead locks, this is best tool. Its displays all the thread details from the java core. The tool clearly identifies the deadlocks. The ThreadAnalyzer can also be used for this.
- IBM Thread and Monitor Dump Analyzer - If your application is taking too much memory, you can generate a heap dump and analyze it with this tool. It even tells you the probable leak suspects.
- IBM Monitoring and Diagnostic Tools for Java™ - Garbage Collection and Memory Visualizer and IBM Pattern Modeling and Analysis Tool for Java Garbage Collector (PMAT) - Basically for the performance issues.You can also see the memory usage over time. It gives good recommendations so that you can change some JVM level garbage collection policies to suite your needs.
Health Center: The above tools are for post analysis. Here is a tool that would help in monitoring a running java application.
Blog by,
Neeraj Krishna and Vijay Raj K ( Oct 13 2008, 09:48:27 AM EDT ) Permalink Comments [0]
WebSphere MQ v6.0.2.5 is now available for Unix and Windows.
WebSphere MQ v6.0.2.5 is available from the web site http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037#1
In addition, the availability of the SupportPacs MQC6 (the WMQ v6 client) and MA0V (the Async SOAP SupportPac), refreshed to WMQ v6.0.2.5 level have been timed with the availability of the Fixpack. The latest refreshes of the SupportPacs are now available from the following URLs.
MQC6 -> http://www-01.ibm.com/support/docview.wss?rs=171&uid=swg24009961&loc=en_US&cs=utf-8
MA0V -> http://www-01.ibm.com/support/docview.wss?rs=171&uid=swg24009880&loc=en_US&cs=utf-8
( Oct 07 2008, 12:29:12 AM EDT ) Permalink Comments [0]WebSphere MQ V7.0 is now available
WebSphere MQ V7.0 is now available !!!!!!The announcement letter is here. One can download using Passport advantage from here
Also read more about MQ V7 in these links:
- IBM WebSphere MQ V7.0 delivers the universal messaging backbone for SOA and Web 2.0 with enhanced ease of use, performance, and resilience
- WebSphere MQ information center
WMQ v6.0.2.4 is now available
WebSphere MQ v6.0.2.4 is now available for Unix, Windows and iSeries.
Please see the Recommended Fixes for Websphere MQ website -> http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037#1 for more information.
( Jun 26 2008, 11:47:56 PM EDT ) Permalink Comments [0]HL7 application integration into IBM Websphere Message Broker V6.0
Healthcare IndustryHealthcare organizations use a large number of discretely-developed IT applications. The exchange of healthcare related data among these systems is required both for administrative and for medical reasons. In an attempt to resolve challenges faced during system integration of healthcare applications, there has been an evolution of messaging standards such as HL7. Many healthcare industries are looking to build solutions by re-using their existing infrastructure. The existing applications need to communicate with each other and share data through a centralized processing unit. HL7 standard is widely used by different healthcare applications to process healthcare data in a standard format.
Message-oriented middleware (message broker or communication server) is being used to deal with some of the integration issues, for example, application integration, translation of code, mapping of events and message routing.
This article aims to describe how to integrate HL7 applications with Websphere Message Broker.
IBM Websphere Message Broker
IBM Websphere Message Broker (WMB) is a business integration product. Business integration is the coordination and co-existence of all the business processes and applications. It involves bringing together the data and process intelligence in the enterprise, and harnessing them so that users can achieve their business goals through their applications.
The use of Message Broker offers a scalable platform that has access to data sent by various applications in the organization. It improves the dependability and ease of access of data and concentrates all available message flows to a single point. Creating a central point for data improves the quality of healthcare service, the integration of data, and the central management of the links.
HL7 application integration into WMB V6.0 - Sample
WMB is used to integrate healthcare applications built on HL7 standard by performing message transformation, enrichment and routing. Even if the existing healthcare applications are based on other languages like .NET languages, C or Java, they can be integrated using WMB.
WMB provides HL7 message set to parse HL7 healthcare messages coming from different healthcare segments. It integrates healthcare applications built on various HL7 segments. One of the segments is Patient information (PID) segment.
Description of the Sample
The HL7 sample demonstrates how users can use the MRM parser to model the PID segment of an HL7 message in XML and TDS formats. It will demonstrate how the PID segment of HL7 message can be translated from TDS to XML message format and vice-versa.
The HL7 sample contains two message flows. The first message flow converts HL7 messages to XML messages (refer Figure 1), and the second message flow converts XML messages to HL7 messages (refer Figure 2). The message set has been created to parse the HL7 messages containing "MSH (message header) and PID (patient information) segment of HL7 Standard" present in TDS format and in XML format (refer Figure 3.).
Figure 1: Message flow which converts HL7 message to XML message
Figure 2: Message flow which converts XML message to HL7 message
Figure 3: Message set to parse HL7 messages (PID segment messages)
HL7 sample artifacts can be downloaded from:
http://www-1.ibm.com/support/docview.wss?uid=swg24014296
Download the HL7.zip file. Please follow the installation instructions as per ia9n.pdf. The HL7 message flow and message set details can be found under the samples gallery.
Benefits of using WMB as middleware
- Integration of healthcare applications helps to reduce the cost of application upgrades and modifications.
- Any healthcare application developed using ERP, SAP, Siebel, etc can be integrated using WMB.
- Cost and problems associated with point-to-point connection can be reduced as it is easy to maintain the connections, migration of systems is easy and also cost involved in creating new connection is less.
An Introduction to WMQ Queue Manager Processes
An
Introduction to WMQ Queue Manager Processes
When a queue manager is created, you get a message something like this:
Observe that 40 default objects are created when a queue manager is created. An MQ object can be a queue, a channel, a namelist, a listener (in WMQ v6.x), a process, etc. Most of these default objects act as templatesfor defining queue manager objects. For example, SYSTEM.DEFAULT.LOCAL.QUEUE is used as a template and its attributes are copied when a new local queue is defined using “DEFINE QLOCAL” command.
During queue manager creation, it starts some of the queue manager processes, which creates these objects, their equivalent files and directories under /var/mqm/qmgrs/<QMGR_NAME> directory, and other supporting directory structures such as queue manager LOG files under /var/mqm/log directory, and at the end the queue manager is ended.
Now, let us start a queue manager using “strmqm TECHTALK”
We will discuss about output lines of “strmqm” a little later.
Now, if you issue:
In the above output, process “amqzxma0” has a parent PID of ‘1’, whereas it is the parent for most of the other processes. This is the first process created and it is called the “Execution Controller”, simply called EC.A little bit of UNIX here. Since “strmqm” exits at the end, this process will eventually be owned by the first UNIX process “init”. (Technically, “init” is not the first UNIX process. First process is pid ‘0’, the scheduler process).
Apart from spawning other processes that are necessary for the functioning of a queue manager, the EC does few more things:
- Establishing connection between the connecting application process and the agent process.
- Periodically checking the health of the WMQ internal processes.
- Cleaning up its children processes in case of queue manager termination (normal and abnormal), etc.
Few tit-bits about agent process:
- Agent is a threaded process and can service 64-applications at a time. Each application is connected to an agent-thread.
- New agent will be created there are more than 64-connection requests.
- The queue manager uses a pool of agent threads and reuses them.
- AgentClassMapand AgentClassLimit<n> are two tuning parameters to control the number of threads an agent can spawn.
How do these various processes (internal qmgr processes, external qmgr processes and qmgr applicaton processes) interact?
They interacting using Inter Process Communication (IPC) facility provided by the Operating System. Pipes, Named pipes, Shared Memory, Mutexes & Semaphores, Messages Queues are 5 IPC mechanisms according to System V standards. Sockets and Streams are the other two. WMQ “largely” uses Shared Memory mechanism to share the information and access to these resources are serialised / controlled using Mutexes and Semaphores.
Figure 1. Shared memory is the preferred technique of communication between two processes.
In WMQ, we mainly use two terms for shared resources – SUBPOOLS and MEMORY SETS. Subpool is a big chunk of memory, which can be further divided into n-number of memory sets. Memory sets are further divided into EXTENTS. And a queue manager will have a number of SUBPOOL for different purposes – IPCC Subpool, QMGR Subpool, APPLICATION Subpool, SYSTEM Subpool, QMPERSISTENT Subpool, etc. IPCC Subpool is the one which is used for communication between an application process and an WMQ process (mostly AGENT process).How are these
resources created and who creates them?
Any application that wants to connect to the IPCC subpool will generate the key by the above technique and puts a request on the IPCC Subpool. This code will be part of the MQCONN( ) API call.
If “shared memory” is the method of communication, we say that the application is connecting to the queue manager in STANDARD bindings. Since the address space is shared between two processes here, the queue manager is prone to corruption if the application corrupts some part of the shared memory.
If you replace the mode of communication with “pipes” / “sockets”, the mode of connection is termed as “ISOLATED” binding. This can be achieved by specifying MQCNO_ISOLATED_BINDING option in MQCONNX( ) call, or by specifying DefaultBindType=ISOLATED in Connection stanza of qm.ini file. This isolates the application address space completely from that of the agent address space. But, it is less efficient compared to STANDARD binding in efficiency.There is one more type of binding, called FASTPATH, where the application directly access the queue manager resources, bypassing the agent process. This provides the lowest level of protection and is configured only for trusted-applications.
Going back to the output of “strmqm” command:
There are two types of messages: Persistent and Non-Persistent. A non-persistent message does not survive a queue manager restart. However, queue manager can not afford losing a persistent message. But, each time a message is PUT or GET, the message will not be written or removed from the queue file on the file system. This would be very expensive in disk I/O. Queue manager keeps a buffer in memory, called queue buffer, which is flushed into the queue file periodically. To ensure that we do not lose persistent messages, each operation with a persistent message is going to be stored in what is called “log buffer” which will be flushed into the LOG files periodically. These log files are created under /var/mqm/log/<QMGR_NAME> directory based on the settings in mqs.ini file (or arguments passed to crtmqm command).
“amqzmur0” and “amqzmuc0” are the two processes that are responsible for logging the transaction information into the log files. These are called restartable (mur0) and critical (muc0) processes respectively. Discussion about the functioning of LOGGER component is beyond the scope of this article. It is sufficient to understand that each and every operation (CONN, OPEN, PUT, GET, etc) gets logged in the WMQ log, and periodically the contents of the log files that are complete are “synced” with the queue file. This sync operation is also called “checkpoint”.
Figure 2. Log records and periodic checkpoint operations.
If you imagine log file as a continuous strip of information, last checkpoint record indicates the “point-of-consistency” between the log file and the queue file. In other words, information till the log file will be safe in the queue file. If a queue manager crashes, when you restart the queue manager, the log records “from” the last checkpoint will be played-forward to bring the queue manager to a state it was when it last crashed. This is how persistent messages are recovered during a crash. Even during queue manager start operation, these log records are played-forward. This is indicated by the messages:You must have already been familiar with the following error:
First of all, an user should belong to “mqm” group (on UNIXes) to be able to perform system administration operations such as creating, starting qmgrs, creating qmgr objects, etc.
This means, one can write their own OAM and make WMQ use these services by making appropriate entries like the one above. It is a kind of pluggable components.
Till now, we have been talking about a single queue manager and an application connecting to a queue manager directly. There are other modes of communication possible – An application connecting to a queue manager over a TCP/IP channel (CLIENT connection), and an application communicating with a remote queue manager through another queue manager.
Figure 3. Application connecting over CLIENT channel or through another queue manager.Application connecting over CLIENT channel or through another queue manager.
In both cases, the receiving queue manager will have to listen on a port (could be TCP/IP or LU6.2, etc). The common listener process is “runmqlsr”. This listens for the incoming requests and hands over the request to a process called “amqrmppa”, also called channel pooler. This is a threaded process and based on the number of requests, more number of “amqrmppa” process will be spawned.
APPLICATION ---> AGENT --->
QMGR RESOURCES
APPLICATION ---> SENDING MCA ---> RECEIVING MCA
(LISTENER/AMQRMPPA) ---> AGENT ---> QMGR RESOURCES
Application
connecting to a queue manager over CLIENT
CHANNEL:
APPLICATION ---> TCP/IP ---> LISTENER/AMQRMPPA ---> AGENT
---> QMGR RESOURCES
Lastly, in situations where more number of queue managers want to communicate among each other, CLUSTERING of queue managers helps the administration overhead by automatically defining channel connections between some of the queue managers.
As you must be aware, a cluster is recommended to have at least TWO full-repository queue managers (Refer Figure below). A full-repository process is going to hold information about the ENTIRE cluster, whereas a partial repository queue manager INITIALLY holds information about itself and over a period of time it builds up the information which is required for its operation. These repository information are stored in the form of messages inside SYSTEM.CLUSTER.REPOSITORY.QUEUE. "amqrrmfa" is the process, called repository process. This process is responsible for retrieving information from the repository queue, updating the information, etc.Figure 4.
Queue manager CLUSTER and REPOSITORY
process, AMQRRMFA.
Although I have not
covered each and every process, these are
the basic things one can start with.
TechTalk Series
Ladies and Gentlemen,
Moving forward you would get to imbibe much with products, WMQ & WMB. A series of TechTalks are being regularly organized in respective areas; Wherein experts in areas would enlighten us with sweeping knowledge - a resultant of their unfathomable learning and extensive experience in areas.
We plan to publish the same material in this Blog. We are sure this series of posts would be of much use to our esteemed audience.
Keeping watching,
IBM-Messaging-Evangelists
WebSphere MQ v7.0 Open Beta announced.
If you are interested in a sneak preview of WebSphere MQ v7.0, here is your chance to get it. The WebSphere MQ v7.0 Open Beta program has been announced. To read more about WebSphere MQ v7.0 and participate in the Open Beta program, please visit this link
You can also download presentations, lab exercises and a draft RedBook at the above link.
There is also a WebSphere MQ V7.0 Open Beta Forum to ask questions, report bugs and just keep track on whats going on. Go ahead and enrol yourself - both to the Open Beta and Open Beta forum
( Apr 04 2008, 12:37:38 AM EDT ) Permalink Comments [1]



