How to choose between Infiniband (IB) or a RoCE Ethernet RDMA capable network type?
When deploying a Db2® pureScale® cluster, whether it is brand new or upgrading a current cluster to a next generation hardware stack, using RoCE Ethernet network is almost always the best choice.
Ethernet is everywhere, and many of the switches are also RoCE capable. This not only increases availability but it often decreases the overall cost, both hardware and human.
- Db2 pureScale has a mix of messages sizes, small messages are used when latency is of the utmost importance.
- Most workloads do not run at 100% network utilization.
As of Db2 version 11.5, QDR remains to be the highest speed IB adapter supported on AIX. On the RoCE front, 40Gb and 100Gb adapters have been added to the supported matrix recently. This has closed the small performance gap between the two networks to a point where RoCE performance meets the performance requirements for most, if not all, OLTP customers’ needs.
As a result, from both the cost and performance perspectives RoCE Ethernet is the recommended network type for even the most demanding production clusters.