Topic
  • 3 replies
  • Latest Post - ‏2013-10-10T13:17:29Z by Pablo_Sanchez
Chris Ahrendt
Chris Ahrendt
1 Post

Pinned topic Routing based on a Comparison to a List

‏2013-10-09T15:21:39Z |

I need help with something that I know can be done… I just need to know how and the most efficient way to do it…
 
I have a Datapower gateway which sits in front of my Events Engine…
I need to do the following to prevent my database from exploding when I send persistent / durable messages to the event engine
(and no I can't fix the DB it's a problem with WBE 7 and the way it works with Oracle)

 
So What I want to do is the following:
 
·         I have a stream of MQ messages coming into the DP pizza box that need to be converted.. and that is already done.. no issues.. been in prod for 2 years.
·         All of these messages (3.5 k tps) are currently non durable / non persistent…  working like a champ…

Now This is what I need to do to the process:
·         Daily send a updated list of account numbers to the DP box to use in a new process.
·         DP will then read the account message off the new queue and store it in an array for use during the day. ßfirst issue where I know it's possible but not how. The DP online manual reads like stereo instructions so no clue how to do this. (this would be aprox 3k array of 25bytes for each acct#)
·         Messages will continue to flow in on another queue from the SOR. ßCurrent process.
·         Messages need to now be compared with one field in the incoming message and compare it to the Array list of accounts.. ßNew
·         If the account matches one in the list  then the process needs to generate 2 messages out of the DP box.. one which goes to the durable queue and the other that goes to the non-durable queue.. The difference being that the durable will have 2 extra fields in its mapping which the non-durable does not. ßNew
 
What is the easiest and least performance impact to do this… according to my offshore contractors they have not done this before. They tried at one point in the past to do dynamic routing but claim the performance hit was a lot.

Can Anyone help?

 

  • kenhygh
    kenhygh
    2007 Posts

    Re: Routing based on a Comparison to a List

    ‏2013-10-09T16:51:41Z  

    Chris,

    I'd have your 'array' in an XML file pushed to the DP box when it gets updated. Have a stylesheet read the incoming message for the account number, look it up in the XML file (which will get cached the first time it's read), and do your logic on transformation and routing. For the non-durable PUT, you could either do it in a conditional with a Results-Asynch, or in a url-open() call in your stylesheet.

  • swlinn
    swlinn
    1395 Posts

    Re: Routing based on a Comparison to a List

    ‏2013-10-09T18:42:13Z  
    • kenhygh
    • ‏2013-10-09T16:51:41Z

    Chris,

    I'd have your 'array' in an XML file pushed to the DP box when it gets updated. Have a stylesheet read the incoming message for the account number, look it up in the XML file (which will get cached the first time it's read), and do your logic on transformation and routing. For the non-durable PUT, you could either do it in a conditional with a Results-Asynch, or in a url-open() call in your stylesheet.

    Hi Chris,

    If your list of account numbers has to be an MQ message instead of an XML file that is pushed as Ken suggests, you could always transform that MQ message to an XML schema and a) use an internal SOMA call to update an on appliance file b) send it out of DP as a PUT to a web server, and your subsequent requests could pull the file, which would be cached assuming you have your caching policy configured, so you can do your lookup.  By durable vs non-durable queue, I assume that is just your destination queue on your backside put, so you should have a dynamic destination for your MPGW and you could set it appropriately for one queue or the other depending upon your lookup.

    Regards,

    Steve

  • Pablo_Sanchez
    Pablo_Sanchez
    89 Posts

    Re: Routing based on a Comparison to a List

    ‏2013-10-10T13:17:29Z  
    • swlinn
    • ‏2013-10-09T18:42:13Z

    Hi Chris,

    If your list of account numbers has to be an MQ message instead of an XML file that is pushed as Ken suggests, you could always transform that MQ message to an XML schema and a) use an internal SOMA call to update an on appliance file b) send it out of DP as a PUT to a web server, and your subsequent requests could pull the file, which would be cached assuming you have your caching policy configured, so you can do your lookup.  By durable vs non-durable queue, I assume that is just your destination queue on your backside put, so you should have a dynamic destination for your MPGW and you could set it appropriately for one queue or the other depending upon your lookup.

    Regards,

    Steve

    Regarding where to store the file, I think you can also use url-open on a NFS mount to read/write/append to an external "xml list" file.

    Not sure whether this would be better than use an internal file, as this file would need to be fetched and cached by DP anyway when doing the comparison.

    (more info on NFS can be found on the extension function catalog: url-open (NFS URLs))