Network requirements and precheck

This topic describes the networking requirements that must be met before using IBM Spectrum Scale Erasure Code Edition.

In the IBM Spectrum Scale Erasure Code Edition configuration, network bandwidth is consumed by the client workload as well as the backend erasure code traffic between nodes. For read I/O, every 1.0 Gbps of usable bandwidth requires 2.0 Gbps of total bandwidth. For write, the overhead depends on the selected erasure code. When writing with 8+3P, each 1.0 Gbps of usable bandwidth requires 2.4 Gbps of total bandwidth. This factor is 2.25 for 8+2P, 2.5 for 4+2P, and 2.75 for 4+3P.

Additional network considerations and requirements are as follows:
  • Linux® bonding is supported on mode 1 (active-backup) for Ethernet and RDMA and mode 4 (IEEE 802.3ad) on Ethernet only. For mode 4 any xmit_hash_policy is supported, however, it is recommended to use layer3+4.
  • Jumbo frames of 9000 MTU (on Ethernet) or higher (on RDMA) is recommended.
  • When using Cluster Export Services protocol software with IBM Spectrum Scale Erasure Code Edition, a dedicated network for CES protocol traffic is required.
Network Key Performance Indicators are listed as follows:
  • The average ICMP latency between any two storage nodes should be 1 msec or less.
  • The maximum ICMP latency between any two storage nodes should be 2 msec or less.
  • The standard deviation should be 0.333 msec or less on the ICMP latency measurements.
  • The minimum throughput test of 2000 MB/sec with 1 client and all the other nodes as server for read test. Note that this is a very specific test, not a performance estimator.
  • The difference between the maximum and minimum throughput values cannot be more than 20%.
  • The ICMP latency metrics should be collected over an extended period, at least 500 seconds for each measurement.
  • The throughput metrics should be collected over an extended period, at least 1200 seconds for each measurement.
Note: You can use the SpectrumScale_NETWORK_READINESS open source tool to check the defined KPI. This tool is available on the IBM Spectrum Scale Tools GitHub (https://github.com/IBM/SpectrumScale_NETWORK_READINESS). Contact IBM® for further details.