Precision of Intermediate Results

Table 76 describes the default precision rules in more detail.

Table 76. Precision of Intermediate Results
Operation Result Precision
Note:
The following operations produce a numeric result. L1 and L2 are the number of digits of the two operands. Lr is the number of digits of the result. D1 and D2 are the number of decimal places of the two operands. Dr is the number of decimal places of the result. T is a temporary value.
N1+N2 T=min (max (L1-D1, L2-D2)+1, 63)

Dr=min (max (D1,D2), 63-t)

Lr=t+Dr

N1-N2 T=min (max (L1-D1, L2-D2)+1, 63)

Dr=min (max (D1,D2), 63-t)

Lr=t+Dr

N1*N2 Lr=min (L1+L2, 63)

Dr=min (D1+D2, 63-min ((L1-D1)+(L2-D2), 63))

N1/N2 Lr=63

Dr=max (63-((L1-D1)+D2), 0)

N1**N2 Double float
Note:
The following operations produce a character result. Ln represents the length of the operand in number of characters.
C1+C2 Lr=min(L1+L2,16773104)
Note:
The following operations produce a DBCS result. Ln represents the length of the operand in number of DBCS characters.
D1+D2 Lr=min(L1+L2,8386552)
Note:
The following operations produce a result of type character with subtype indicator. The result is always an indicator value (1 character).
V1=V2 1 (indicator)
V1>=V2 1 (indicator)
V1>V2 1 (indicator)
V1<=V2 1 (indicator)
V1<V2 1 (indicator)
V1<>V2 1 (indicator)
V1 AND V2 1 (indicator)
V1 OR V2 1 (indicator)


[ Top of Page | Previous Page | Next Page | Contents | Index ]