Performance of decimal overflows

The optimizer in Enterprise COBOL 6 was designed to truncate values at the earliest opportunity, provided that early truncation would not remove digits that would affect the remainder of a computation. This sometimes allowed the code generator to use smaller data types or avoid generating instructions that would not have any impact on the final result. Truncating in the middle of a computation is often done with packed-decimal instructions on the hardware, which can generate decimal overflows.

Normally, Enterprise COBOL programs do not set or alter the hardware's overflow mask. Without the overflow mask being set, an overflow is suppressed at the hardware level. However, certain COBOL features or other languages in an Interlanguage communication (ILC) application may set the overflow mask so that a hardware exception is not suppressed when an overflow occurs. For details, see Decimal overflow implications in ILC applications in the Enterprise COBOL Performance Tuning Guide. When the unsuppressed overflow exception occurs in a COBOL program, the LE condition handler is invoked, and consequently the exception is ignored, as COBOL programs should not overflow. But the time taken to invoke LE and transfer control back to the COBOL program decreases performance.

In addition, for ARCH(10) and ARCH(11) with the availability of the decimal floating point (DFP) facility, and ARCH(12) with the vector decimal facility, the compiler uses the truncation capabilities of these facilities to further improve performance. However, this can lead to more potential overflows than previous architecture levels (of COBOL 4.2 and earlier compilers), and thus lead to performance issues as described earlier. The changes in COBOL 6 to truncate earlier and using different instructions do not change the behavior of Enterprise COBOL 6 programs compared to Enterprise COBOL 4. The only impact is the possible performance issues, and those only affect ILC applications.

Starting with ARCH(13) in COBOL 6.3, the compiler uses the vector decimal Instruction Overflow Mask (IOM) to suppress decimal overflows at the instruction level independently of the program mask setting. Therefore, COBOL performance is unaffected by overflows, even in ILC applications.