Precision of Intermediate Results
Table 1 describes the default precision rules in more detail.
Operation | Result Precision |
---|---|
Note: The following operations produce a numeric result. L1 and L2 are the number
of digits of the two operands. Lr is the number of digits of the result. D1
and D2 are the number of decimal places of the two operands. Dr is the number
of decimal places of the result. T is a temporary value.
|
|
N1+N2 | T=min (max (L1-D1, L2-D2)+1, 63)
Dr=min (max (D1,D2), 63-t) Lr=t+Dr |
N1-N2 | T=min (max (L1-D1, L2-D2)+1, 63)
Dr=min (max (D1,D2), 63-t) Lr=t+Dr |
N1*N2 | Lr=min (L1+L2, 63)
Dr=min (D1+D2, 63-min ((L1-D1)+(L2-D2), 63)) |
N1/N2 | Lr=63
Dr=max (63-((L1-D1)+D2), 0) |
N1**N2 | Double float |
Note: The following operations produce a character result. Ln represents the length
of the operand in number of characters.
|
|
C1+C2 | Lr=min(L1+L2,16773104) |
Note: The following operations produce a DBCS result. Ln represents the length of
the operand in number of DBCS characters.
|
|
D1+D2 | Lr=min(L1+L2,8386552) |
Note: The following operations produce a result of type character with subtype indicator.
The result is always an indicator value (1 character).
|
|
V1=V2 | 1 (indicator) |
V1>=V2 | 1 (indicator) |
V1>V2 | 1 (indicator) |
V1<=V2 | 1 (indicator) |
V1<V2 | 1 (indicator) |
V1<>V2 | 1 (indicator) |
V1 AND V2 | 1 (indicator) |
V1 OR V2 | 1 (indicator) |