Topic
7 replies Latest Post - ‏2009-10-01T19:33:39Z by SystemAdmin
SystemAdmin
SystemAdmin
289 Posts
ACCEPTED ANSWER

Pinned topic persistence of request messages?

‏2009-09-10T02:09:26Z |
Hello, all,

I have a couple of questions about the behavior of WESB.

Q1.
Is there a way or a mechanism to check if the mediation module is working?, like a watchdog or something. How can I check the health of mediation flows?

Q2.
How can I understand the persistence of request messages?
For example, when a mediation flow (or WAS) goes down during the process, the request message will gone? or be stored in somewhere? Is this applied to WebSphere adapters?
My guess is that it depends on the transaction scope.
But there should be a case that we can't include it in one transaction. Take MQ export, for instance, I learned it can't be included in the transaction because it's asynch.

Many thanks in advance,
Updated on 2009-10-01T19:33:39Z at 2009-10-01T19:33:39Z by SystemAdmin
  • kolban
    kolban
    8 Posts
    ACCEPTED ANSWER

    Re: persistence of request messages?

    ‏2009-09-10T04:57:03Z  in response to SystemAdmin
    A mediation flow is deployed as a Java EE application. As such, you can ask WESB/WPS/WAS if the associated application is started or stopped. If started, then it is running. Because the mediation flow is developer designed, WESB can't tell if it is functionally correct ... but if it is installed, we can certainly tell if it is running and open for business.

    WESB aims to transform data and route incoming messages. The persistence of messages is a function of the technology used to send the request and the technology used to send onwards the further requests. For example, a REST request over HTTP is never persisted. If the request is received by WESB and the lights go out in the machine room, WESB can never recover that request as the request from the HTTP client will most certainly be lost. If you must assure delivery, then the current solution is to use asynchronous assured delivery messaging such as JMS or MQ.

    WESB doesn't host processes. WPS does. WESB takes incoming requests, optionally transforms their content and optionally forwards the request on for processing down the line optionally using the content of the request to determine the destination to which the original request should be sent. In messaging, such as JMS and MQ, persistence is the concept of hardening messages in queues such that if the queuing subsystem goes away will the message still be in the queue when the subsystem is restored. Don't confuse this with transactionality which assures that a message will be processed correctly or not at all. When a message arrives at WESB in a queue and then forwarded onwards to another queue, my understanding is that both the receive from the original queue and the send to the target queue can be bracketed by a transaction so that either the receive/send will both happen or neither will happen and the message will remain available for processing.
    • SystemAdmin
      SystemAdmin
      289 Posts
      ACCEPTED ANSWER

      Re: persistence of request messages?

      ‏2009-09-10T08:21:51Z  in response to kolban
      Thank you very much for your help, kolban.

      I'd like to know about your information in greater detail.

      >
      >the current solution is to use
      >asynchronous assured delivery messaging
      >such as JMS or MQ.
      >

      Is it about end-applications?
      Once WESB receives a message from MQ, and then, unfortunately, the application suddenly goes down during the process, the message will be lost, right?

      I know messages are sent and received via SIBus when each SCA component is bound asynchronously and I understand there should be several queues on SIBus between SCA components including mediation modules.
      In this case, the message will stay in a queue if, for example, WAS will be re-started?

      Thanks,
      • kolban
        kolban
        8 Posts
        ACCEPTED ANSWER

        Re: persistence of request messages?

        ‏2009-09-10T14:28:32Z  in response to SystemAdmin
        I'm not sure I fully understand the question ....

        When a message is received from a queue, the receipt of the message can be transactional. What this means is that if I retrieve the message and fail before processing that message properly, everything I did with the message up to the failure is rolled back and the message is returned to the queue for reprocessing. It is as though I never received the message in the first place. This is true for both MQ and SIBus.
        • SystemAdmin
          SystemAdmin
          289 Posts
          ACCEPTED ANSWER

          Re: persistence of request messages?

          ‏2009-09-11T00:29:21Z  in response to kolban
          Hi, kolban,

          Well, I want to know if request messages through MQ, JMS, and HTTP will be persisted within WESB when it goes down, is it similar to persistent messages in MQ?

          In the case of MQ export, when WESB rolls back the transaction, will the message be rolled back to the queue on QMgr?
          I doubt this because QMgr can't be a part of transaction as it's asynch message binding. Then, where is the message?

          BTW, when WESB commits MQGET?
          Regards,
          • kolban
            kolban
            8 Posts
            ACCEPTED ANSWER

            Re: persistence of request messages?

            ‏2009-09-11T04:33:07Z  in response to SystemAdmin
            Thanks for coming back with clarification questions. This is the only way we all learn and I have as much time as anyone would like to help us all understand these products better.

            The bottom line is that if WESB goes down, anything in-flight will not be persisted within WESB. Instead, try and picture a transaction that starts when the message arrives. WESB will take the message from the queue (JMS or MQ) but ... and here is the key thing ... the message provider will not destroy the message from the queue because there is still an outstanding transaction. If something goes wrong, the message appears back in the original queue. This can only work with data that comes from transactional sources such as JMS or MQ ... and not from non-transactional sources such as HTTP.

            Now ... as to MQ Qmgr and your thinking that it can't be part of a transaction. Imagine a message on a queue ... imagine an application getting that message ... and now imagine the application failing. The message will be returned to the queue because the MQGET was transactional. Where messaging becomes non-transactional is when a message is succesfully put ... at that point the transaction that puts the message completes ... and it must be another transaction that performs a get.

            But a transaction can include a:

            "GET ... do-something .... PUT"

            In this case the GET, the something and the PUT are all under one transaction.

            My understanding of when WESB commits an MQGET (i.e. an Export component) is when the Mediation Flow commits.
            • SystemAdmin
              SystemAdmin
              289 Posts
              ACCEPTED ANSWER

              Re: persistence of request messages?

              ‏2009-09-11T05:01:18Z  in response to kolban
              Thank you so much for your comments, kolban.

              It seems that we should consider using JMS/MQ instead of HTTP when messages are critical.
              But one problem is user requirements because there should be a system which doesn't support JMS/MQ.
              Well,,, there're unlimited considerations in designing the architecture...

              From now it's my turn.
              I'll try some hands-ons to clarify it more precisely.
              Let me just start with building the environment and test scenarios...

              Commets are welcome! Always!!

              Cheers,
              • SystemAdmin
                SystemAdmin
                289 Posts
                ACCEPTED ANSWER

                Re: persistence of request messages?

                ‏2009-10-01T19:33:39Z  in response to SystemAdmin
                If you have sytems that do not suppport MQ or JMS, but you have MQ in your architecture, you might want to consider using MQ IPT(internet pass thru - a free support pack with can intercept HTTP requests and convert them to MQ and vice-versa).