Topic
  • 9 replies
  • Latest Post - ‏2014-01-02T18:55:58Z by PPotkay
SKG
SKG
46 Posts

Pinned topic Need performance data for DP XI52

‏2013-12-30T11:54:15Z |

I was unable to find any IBM site that lists the performance benchmarking data for XI52 such as tps handled for different data types, on different protocols, with/without ssl. Can anyone point me to such data ?

  • kenhygh
    kenhygh
    2087 Posts

    Re: Need performance data for DP XI52

    ‏2013-12-30T12:36:53Z  

    Any benchmark data is kept internal to IBM and is not published.

  • SKG
    SKG
    46 Posts

    Re: Need performance data for DP XI52

    ‏2013-12-31T07:56:29Z  

    I asked this query because being a WMB person I can see performance reports for broker being published by IBM here : http://www-01.ibm.com/support/docview.wss?rs=171&uid=swg27007150&loc=en_US&cs=utf-8&lang=en#2, hence if there for WMB why not for DP

  • ted.jump
    ted.jump
    473 Posts

    Re: Need performance data for DP XI52

    ‏2013-12-31T12:26:09Z  
    • SKG
    • ‏2013-12-31T07:56:29Z

    I asked this query because being a WMB person I can see performance reports for broker being published by IBM here : http://www-01.ibm.com/support/docview.wss?rs=171&uid=swg27007150&loc=en_US&cs=utf-8&lang=en#2, hence if there for WMB why not for DP

    No clue why but it's not published.

    Hermann has done some simple evals and shared information here on the forum, recently too, if you look. Doesn't cover a wide range of stuff but does give an indication.

    Similarly I've done some trivial testing with a pair of the XI52 appliances where I work, using an internal loopback, a transform/routing to the second appliance which was using internal loopback, and a transform/routing via 1st to 2nd and via 2nd to an ILOG JRules 7 back-side. The two appliances only talk via HTTPS. The 2nd appliance talks to JRules over HTTP.

    The first two tests were easy to drive beyond an average of 1000 transactions per second, the 3rd was limited to the JRules performance. A trivial JRules app wasn't hard to get an average of 600-800 tps through all the hops. Note that talking directly to the JRules execution unit didn't get any notably better performance, saving at most 2 ms in my testing.

    One thing I've noticed as a quirk of testing is that so far the harder the DP appliance is pushed the more consistently even and good the performance. As in, it's easy to find the processing overhead added by the DP hops holding below 3 ms per appliance, often effectively un-measurably low. Low frequency calls, and one-offs, often see more DP introduced latency.

    My unproven thesis for one of the contributors to this behavior is that DP will discard some logging in favor or processing when  under stress. This affects the output to our Syslog-NG targets (DP will eventually send a record to Syslog stating how many log events were dropped). I also believe that it's possibly affecting the WSM (ITCAM-monitored) data as the WSM record buffer appears fixed-size and includes a drop-count. The latter is required because there's no guarantee on whether ITCAM is logging in frequently enough to query/dump the buffer.

  • PPotkay
    PPotkay
    140 Posts

    Re: Need performance data for DP XI52

    ‏2013-12-31T12:29:18Z  

    I am new to DP and familair with WMQ and WMB and found myself asking the same thing. You would think IBM would be keen on demonstrating and documenting how much horsepower these things have. And allowing us to compare (by reviewing official docs written by people who know what they are doing) the performance of  the exact same workload between firmware versions on the same hardware model, same firmware but different generation hardware, same firmware but physical versus virtual.

    Excluding the virtual appliances, there 's no need for the standard disclaimers of your milage may vary based on the metal hardware and O/S you decide to deploy to - IBm controls all aspects of it! Seems like the perfect scenario for Performance Reports to be created.

     

  • kenhygh
    kenhygh
    2087 Posts

    Re: Need performance data for DP XI52

    ‏2013-12-31T13:15:47Z  
    • PPotkay
    • ‏2013-12-31T12:29:18Z

    I am new to DP and familair with WMQ and WMB and found myself asking the same thing. You would think IBM would be keen on demonstrating and documenting how much horsepower these things have. And allowing us to compare (by reviewing official docs written by people who know what they are doing) the performance of  the exact same workload between firmware versions on the same hardware model, same firmware but different generation hardware, same firmware but physical versus virtual.

    Excluding the virtual appliances, there 's no need for the standard disclaimers of your milage may vary based on the metal hardware and O/S you decide to deploy to - IBm controls all aspects of it! Seems like the perfect scenario for Performance Reports to be created.

     

    I don't disagree, but I don't make these decisions. Your best bet is probably to try to get this through your IBM sales rep.

  • SKG
    SKG
    46 Posts

    Re: Need performance data for DP XI52

    ‏2014-01-02T05:35:09Z  
    • PPotkay
    • ‏2013-12-31T12:29:18Z

    I am new to DP and familair with WMQ and WMB and found myself asking the same thing. You would think IBM would be keen on demonstrating and documenting how much horsepower these things have. And allowing us to compare (by reviewing official docs written by people who know what they are doing) the performance of  the exact same workload between firmware versions on the same hardware model, same firmware but different generation hardware, same firmware but physical versus virtual.

    Excluding the virtual appliances, there 's no need for the standard disclaimers of your milage may vary based on the metal hardware and O/S you decide to deploy to - IBm controls all aspects of it! Seems like the perfect scenario for Performance Reports to be created.

     

    Good to see you here Mr. Potkay after mqseries.net, if you are the same person.

    Updated on 2014-01-02T10:14:25Z at 2014-01-02T10:14:25Z by SKG
  • SKG
    SKG
    46 Posts

    Re: Need performance data for DP XI52

    ‏2014-01-02T10:13:22Z  
    • ted.jump
    • ‏2013-12-31T12:26:09Z

    No clue why but it's not published.

    Hermann has done some simple evals and shared information here on the forum, recently too, if you look. Doesn't cover a wide range of stuff but does give an indication.

    Similarly I've done some trivial testing with a pair of the XI52 appliances where I work, using an internal loopback, a transform/routing to the second appliance which was using internal loopback, and a transform/routing via 1st to 2nd and via 2nd to an ILOG JRules 7 back-side. The two appliances only talk via HTTPS. The 2nd appliance talks to JRules over HTTP.

    The first two tests were easy to drive beyond an average of 1000 transactions per second, the 3rd was limited to the JRules performance. A trivial JRules app wasn't hard to get an average of 600-800 tps through all the hops. Note that talking directly to the JRules execution unit didn't get any notably better performance, saving at most 2 ms in my testing.

    One thing I've noticed as a quirk of testing is that so far the harder the DP appliance is pushed the more consistently even and good the performance. As in, it's easy to find the processing overhead added by the DP hops holding below 3 ms per appliance, often effectively un-measurably low. Low frequency calls, and one-offs, often see more DP introduced latency.

    My unproven thesis for one of the contributors to this behavior is that DP will discard some logging in favor or processing when  under stress. This affects the output to our Syslog-NG targets (DP will eventually send a record to Syslog stating how many log events were dropped). I also believe that it's possibly affecting the WSM (ITCAM-monitored) data as the WSM record buffer appears fixed-size and includes a drop-count. The latter is required because there's no guarantee on whether ITCAM is logging in frequently enough to query/dump the buffer.

    Configured my machine to act as a client and fired requests onto a DP XI52 box configured in loopback mode. Used JMETER and at 1000 users (threads) 0 rampup and loop forever was getting a throughput > 53000/min. At 2000 users this was > 52000/min

  • SKG
    SKG
    46 Posts

    Re: Need performance data for DP XI52

    ‏2014-01-02T13:39:04Z  

    On googling I only got this ..........http://www.omg.org/news/whitepapers/sponsors-wp/Intel_SOAE_DataPower_Perf_Compare_White_Paper.pdf, even though this comparison is with xi50 but the stats are not believable. Some guy was claiming that DP on average can handle 3000 tps and can go upto 5000 tps but these figures are way lower.

  • PPotkay
    PPotkay
    140 Posts

    Re: Need performance data for DP XI52

    ‏2014-01-02T18:55:58Z  
    • SKG
    • ‏2014-01-02T05:35:09Z

    Good to see you here Mr. Potkay after mqseries.net, if you are the same person.

    Yup its me. My career has added DataPower to my list of responsabilities, so I'll be asking more than answering initially.