Troubleshooting
Problem
This document provides information on a Report Assistant (Web Query 1.1.x) or an InfoAssist (Web Query 1.1.2 and later) report using a DEFINE decimal type calculated field that is not rounded as expected.
Resolving The Problem
A Report Assistant or an InfoAssist report using a DEFINE decimal type calculated field is not rounded as expected.
Here is an example:
BPQx.045 and BQPx.025 are both defined at Decimal 12.2 (D12.2).
The last column of the report is defined as BPQx.045 + BQPx.025 and is also Decimal 12.2.
Line 2 and 3 of the report show the rounding error as the total is not equal to the sum of the two columns.
ie for Line 2, 27.86 + 15.48 <> 43.33
The behavior of Decimal and Packed columns are different.
Changing the decimal columns to packed resolved the rounding problem.
The reason for this is that Doubles round their display value; however, they do not round their actual value.
Because all computes are done with floating point math, Packed fields are converted to DEFINEs internally; that should enforce rounding at the value level.
Historical Number
498392042
Was this topic helpful?
Document Information
Modified date:
11 November 2019
UID
nas8N1013487