Cluster Contention - default block size

Edit online

This test shows the impact on transaction throughput when reducing the default block size from 8 KB to 4 KB.

The results from Cluster file system OCFS2 versus Oracle Automatic Storage Management showed that a large default database block size might be a drawback when using Oracle RAC with shared tables, because the global cache is managed in units of the database block size. Large blocks cause higher traffic for the interconnect, and might increase the chances of a lock collision.

Figure 1 shows the impact on transactional throughput and wait classes when decreasing the default block size from 8 KB to 4 KB.
Figure 1. Impact of decreasing the database block size on transaction throughput and time spent in wait events
Bar graph of the impact of decreasing the database block size (from 8 KB to 4 KB) on transaction throughput and time spent in wait events. The x-axis has four bars (TPM, Cluster, Commit, System I/O). The y-axis has the impact in percentage, ranging from -8% to +6%. For the four bars, the values are: +4%, -4.5%, -6.5%, and +5.2% respectively.

Observation

When the Oracle database default block size was decreased from 8 KB to 4 KB, the transaction throughput improved by 4%. The data shown describes classes of waits in which cluster commit and System I/O are the most significant portions. There was 4.71% less waiting due to cluster contention compared to the 8 KB block size. The commit contention was reduced by 6%. The system I/O as a percent of total activity went up 5%, corresponding to the increased throughput, which causes more log file I/O and data updates.

Figure 2 shows the impact on the global cache profile when decreasing the database block size from 8 KB to 4 KB .
Figure 2. Impact of decreasing the database block size on the global cache profile
Bar graph of the impact on the global cache load profile when decreasing the database block size from 8 KB to 4 KB. The x-axis has five bars: (1) Estimated interconnect traffic in KB, (2) GCS/GES messages sent and received, (3) Global cache blocks served and received, (4) CR blocks received, and (5) Current Blocks received. The y-axis has the impact in percentage, ranging from -50% to +10%. For the five bars, the values are: -47%, +4%, +2%, -8%, and -15% respectively.

Observation

When the database default block size was reduced from 8 KB to 4 KB, the amount of traffic over the interconnect to perform cache fusion went down by 46%. The Global Cache Load Profile from the AWR reports from both instances gives a breakdown of the types of traffic, which is composed of both data blocks and messages. Shown here is the increase in both types of data. GCS/GES messages increased 4% in units of messages sent and received, and Global Cache blocks sent and received increased 2%. GES stands for Global Enqueue Services and GCS stands for Global Cache Services, both are important for performance of the cache system.

CR Blocks Received went down 8%, and Current Blocks Received went down by 15% The distinction between CR blocks and Current Blocks is whether they are consistent reads across the cluster or Current or "as is" for the latest.

Conclusion

The reduction of the database block size leads, as expected, to a decrease of the interconnect traffic of approximately one-half. The transaction throughput follows the cluster wait events (less contention means more throughput), while the commit contention is reduced even more, which indicates a reduced lock contention. The quantity of messages sent and received increase as required by the higher throughput, but the number of data block is reduced. This reduction clearly shows that the contention on the blocks has been reduced, because locks for the two different 4 KB blocks in the same 8 KB block are now two independent locks, making the sending of the block unnecessary (but not the communication that a lock must be placed).

The performance improvement as measured by transaction throughput leads to the recommendation of using the smallest possible block size (such as 4 KB), as default in an Oracle RAC system with shared tables.