Topic
10 replies Latest Post - ‏2013-06-12T20:41:09Z by fjb_saper
SystemAdmin
SystemAdmin
8523 Posts
ACCEPTED ANSWER

Pinned topic JMS Messages : Files Stuck in Transmission Queue, failing to reprocess

‏2013-03-25T11:32:01Z |
Hi Folks,

We have an issue going around JMS Messages sent on MQ Ver 6.0 Server running on AIX-Platform.

We need to understand the standard procedure to be followed in extracting and re-processing the JMS messgaes with all MQ headers ( Transmission Queue header ) intact. Currently we are making use RFHUtil tool for extraction and reprocessing the messages.

PS: The issue surfaces when we receive the files of size greater than 60 MB in data from the client.

Also we are facing a situation where over a network there are loss of certain packets, though the data transfer is happening through a dedicated channel. Need to know the best approach/method to be followed in tracking these lost packets.

Thanks in advance for your co-operation.

Cheers,
Hari,
Updated on 2013-04-03T11:03:31Z at 2013-04-03T11:03:31Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    8523 Posts
    ACCEPTED ANSWER

    Re: JMS Messages : Files Stuck in Transmission Queue, failing to reprocess

    ‏2013-03-26T01:55:59Z  in response to SystemAdmin
    >We need to understand the standard procedure to be followed in extracting and re-processing
    >the JMS messgaes with all MQ headers ( Transmission Queue header ) intact.

    What do you mean by reprocess?

    Once any type of message with Transmission Header(MQXQH) is sitting on a transmission queue, it is the responsibility of MQ to deliver the message to the intended destination. There is no need to reprocess anything, MQ should deliver it eventually.
    >Also we are facing a situation where over a network there are loss of certain packets,
    >though the data transfer is happening through a dedicated channel. Need to know the
    >best approach/method to be followed in tracking these lost packets.

    Are you seeing TCP errors in the Queue Manager error log on the MQ channel? Lost packets should be investigated by the Network support team. MQ can't handle lost packets, nor is it MQ's responsibility, it assumes the network has sufficient reliability.

    HTH, G.
    • SystemAdmin
      SystemAdmin
      8523 Posts
      ACCEPTED ANSWER

      Re: JMS Messages : Files Stuck in Transmission Queue, failing to reprocess

      ‏2013-03-26T09:48:57Z  in response to SystemAdmin
      Hi GBaddeley ,

      Thanks for the below response. Let me just explain the scenario in more detail.

      We found there were around 1000+ files stuck in Tx queue. There were few files of 80 MB data which were preventing other files with less data size from processing and there was a time delay which can't be afforded.
      So we had to remove those 80 MB files from the queue to allow other files to process. Once this was done we tried to reprocess those files of 80 MB data size, but unfortunately it didn't work out as expected. The Tx Queue error log said " (RC2260): MQRC_XQH_ERROR ..." So need to know if we have any standard mechanism to follow when processing files of huge data size.

      Coming to TCP Network issues, your post was helpfull. Thanks

      Cheers,
      Hari.
      • SystemAdmin
        SystemAdmin
        8523 Posts
        ACCEPTED ANSWER

        Re: JMS Messages : Files Stuck in Transmission Queue, failing to reprocess

        ‏2013-03-26T11:29:31Z  in response to SystemAdmin
        If you MQGET a message directly off a transmission queue, the message will contain an MQXQH header at the front of it which was placed on there by the queue manager when you originally MQPUT the message and it resulted in being placed on a transmission queue due to needing to go over a channel.

        An MQXQH (transmission header) looks like this:-
        struct tagMQXQH {
        MQCHAR4 StrucId; /* Structure identifier */
        MQLONG Version; /* Structure version number */
        MQCHAR48 RemoteQName; /* Name of destination queue */
        MQCHAR48 RemoteQMgrName; /* Name of destination queue manager */
        MQMD1 MsgDesc; /* Original message descriptor */
        };

        If you intend to MQPUT these messages onto another queue, you should first strip this header off. The intention is that you should use it to see which queue the message is targeted for. MQRC_XQH_ERROR tells you that you tried to do an MQPUT of a message but the length wasn't long enough to include the whole MQXQH which given that your message is 80MB is a surprise.

        Can you tell us any more about how you are re-putting these messages?

        There are also tools that can do this for you - for example MO71 which also gives you the option to strip the headers off as you move the message from one queue to another. However, in this case you have already taken them off so I guess it is of less use.

        Cheers
        Morag
        • SystemAdmin
          SystemAdmin
          8523 Posts
          ACCEPTED ANSWER

          Re: JMS Messages : Files Stuck in Transmission Queue, failing to reprocess

          ‏2013-03-26T16:28:34Z  in response to SystemAdmin
          Hi Morag,

          Thanks for your response. Let me explain you the process we have followed here.

          1. RFHUTIL tool is used to extract the message of around 80 MB data from Xmit Q. This was performed to allow other messages of less data size to process further to destination.

          2. Post completion of above step , RFHUTIL tool was again used to load the message of 80 MB data back to Xmit Q. We haven't stripped of MQXQH as the messages were loaded back to Xmit Q to allow it to process further. During this point we got MQRC_XQH_ERROR.

          Ideally we had expected this message to process, but it didn't happen. So we are not sure on the failure points here. So we need to know how this can be made to work now.

          Cheers,
          Hari.
          • SystemAdmin
            SystemAdmin
            8523 Posts
            ACCEPTED ANSWER

            Re: JMS Messages : Files Stuck in Transmission Queue, failing to reprocess

            ‏2013-03-26T17:20:25Z  in response to SystemAdmin
            I suspect we need to be able to see how the MQPUT was coded then, and for that you will need to engage the owner of RFHUtil. You may be able to discover the error with MQ trace, but you will still need his assistance to fix it.
            • SystemAdmin
              SystemAdmin
              8523 Posts
              ACCEPTED ANSWER

              Re: JMS Messages : Files Stuck in Transmission Queue, failing to reprocess

              ‏2013-03-26T22:34:19Z  in response to SystemAdmin
              >We found there were around 1000+ files stuck in Tx queue. There were few files of 80 MB data which were
              >preventing other files with less data size from processing and there was a time delay which can't be afforded.
              >So we had to remove those 80 MB files from the queue to allow other files to process. Once this was done we tried to
              >reprocess those files of 80 MB data size, but unfortunately it didn't work out as expected. The Tx Queue error log
              >said " (RC2260): MQRC_XQH_ERROR ..." So need to know if we have any standard mechanism to follow when processing files
              >of huge data size.

              If you are going to move messages off a XMITQ and then put them back on at a later time, it is not necessary to do anything with the headers. Just leave the entire Message Descriptor and Message Data payload as they were. The MQMD.Format needs to be preserved as "MQXMIT ". I'm not if the Message Channel Agent checks other MQMD fields.

              I suggest that if you have a wide range of message sizes and priorities going to a remote queue manager that you set up dedicated transmission queues (eg. QMDEST1 for normal traffic, QMDEST1_BIG for large messages) and dedicated sender channels to provide different "classes" of MQ message transport service.

              HTH, G.
              • Cosmic_HV
                Cosmic_HV
                8 Posts
                ACCEPTED ANSWER

                Re: JMS Messages : Files Stuck in Transmission Queue, failing to reprocess

                ‏2013-04-23T08:38:14Z  in response to SystemAdmin

                Hi  GBaddeley,

                Hope you are doing great !

                Thanks for the reply. I am working on that further. Basically we have 2 items on the radar to look at .

                1. Need to understand if we have a standard procedure to take back up of the messages from XMITQ.

                2. Reprocessing files whose file sizes may vary betwenn 0-80 MB .

                Let me know if we have any inforamtion on the above to drive this further

                Cheers,

                Hari

                 

                 

                 

  • SystemAdmin
    SystemAdmin
    8523 Posts
    ACCEPTED ANSWER

    Re: JMS Messages : Files Stuck in Transmission Queue, failing to reprocess

    ‏2013-04-03T11:03:31Z  in response to SystemAdmin
    Hello,

    Also please check if the remote queue manager can handle messages that large. I know that the maximum limit is 100MB which should be a problem, but it's ok to double check the qmgr definition just to be sure.

    Thanks
    • Cosmic_HV
      Cosmic_HV
      8 Posts
      ACCEPTED ANSWER

      Re: JMS Messages : Files Stuck in Transmission Queue, failing to reprocess

      ‏2013-05-02T13:33:12Z  in response to SystemAdmin

      Hi,

      Thanks for the response,

      Yes, the QMGR is configured to handle the files upto 100MB !

      Cheers,

      Hari

       

  • fjb_saper
    fjb_saper
    167 Posts
    ACCEPTED ANSWER

    Re: JMS Messages : Files Stuck in Transmission Queue, failing to reprocess

    ‏2013-06-12T20:41:09Z  in response to SystemAdmin

    You probably should have 2 channels. One for handling large messages with a batchsize of 1.

    One for handling regular size messages with a bachsize of 50?

    This will allow you to process regular size messages... Alternatively you could also have the transmission queue set to priority delivery mode and have all regular size messages have  higher priority than large size message... however the dual channel solution is the better one.