Benchmarking and systems performance - hosted by Elisabeth Stahl

Elisabeth Stahl

Blog Authors:  ELISABETH STAHL   is Chief Technical Strategist and Executive IT Specialist, IBM Systems and Technology Group, and has been working in systems performance for over 25 years.

  • Add a Comment
  • Edit
  • More Actions v
  • Quarantine this Entry

Comments (6)

1 localhost commented Permalink

TPC-C was developed and introduced back in 1992, before the internet even took form and SMP systems was in its infancy. It is now EOLed and replaced with TPC-E. <div>&nbsp;</div> Why is it that companies like IBM still run this benchmark when it doesn't represent any real workload today? 4TB of RAM is what got IBM these results. Not 5GHz CPUs. When larger dimm sizes come out, you can bet that the new TPC-C scores will come out.<div>&nbsp;</div> So why brag about leading a 16 year benchmark? This does customers no service. <div>&nbsp;</div> If IBM wants bragging rights, then publish benchmarks that are relevant to the high end SMP community. Benchmarks like TPC-E (no 595 results), TPC-H (no Power6 595 results) would show who really is king. Otherwise, its just Benchmarketing at its best. All those basing purchasing decisions on TPC-C are just fools.

2 localhost commented Permalink

Phil, Thanks for your comments. Despite TPC-C's age, it is still an important OLTP benchmark in the whole portfolio of available benchmarks and everyone wants to see results on it. I agree that it will also be great to see different technologies published on the TPC-E benchmark in the future. See http://www.ibm.com/systems/power/hardware/benchmarks for the wide variety of benchmarks where IBM Power Systems have leadership.

3 localhost commented Permalink

Elisabeth,<div>&nbsp;</div> I would appreciate your comment on this statement in a recent IBM press release from March 2008:<div>&nbsp;</div> "Many IT professionals consider TPC-C to be a valid indicator of "real-world" OLTP system performance."http://www-03.ibm.com/press/us/en/pressrelease/23722.wss<div>&nbsp;</div> This IBM paper comes to different conclusion:ftp://ftp.software.ibm.com/eserver/benchmarks/wp_TPC-E_Benchmark_022307.pdf<div>&nbsp;</div> Thanks in advance.<div>&nbsp;</div> <div>&nbsp;</div> <div>&nbsp;</div>

4 localhost commented Permalink

Hi Jim, Thanks for your comments. It is interesting how both of these discussions on TPC-C can be valid at this point in time. I know that many professionals in the IT world do consider TPC-C to be an important OLTP benchmark and still rely on it as one data point when making decisions. I also agree that it is important to "renovate" and that is why TPC-E is so important as well. As described in the TPC-E paper, we are now in that time where there is overlap between these benchmarks and both are valuable. At some point in the future, IT professionals may cease to request TPC-C, and TPC-E may become the standard.

5 localhost commented Permalink

Thanks for your reply. <div>&nbsp;</div> I notice that AIX 5.3 was used for this benchmark instead of AIX 6.1 which was used for the SAP SD benchmarks. <div>&nbsp;</div> Can you give us some insight on why this was done?

6 localhost commented Permalink

Jim - As we know, TPC-C takes a long time to plan and setup. The team started with AIX 5.3 and didn't want to switch in the middle. We should be seeing more benchmarks with 6.1 in the future. Thanks for your question !

Add a Comment Add a Comment