Calculating scale factors

If you create a spatial reference system and any of the coordinates that you are working with are decimal values, calculate the appropriate scale factors for your coordinates and measures. Scale factors are numbers that, when multiplied by decimal coordinates and measures, yields integers with at least the same number of significant digits as the original coordinates and measures.

To calculate the scale factors:

1. Determine which X and Y coordinates are, or are likely to be, decimal numbers.
   For example, suppose that of the various X and Y coordinates that you will be dealing with, you determine that three of them are decimal numbers: 1.23, 5.1235, and 6.789.
2. Find the decimal coordinate that has the longest decimal precision. Then determine by what power of 10 this coordinate can be multiplied to yield an integer of equal precision.
   For example, of the three decimal coordinates in the current example, 5.1235 has the longest decimal precision. Multiplying it by 10 to the fourth power (10000) yields the integer 51235.
3. Determine whether the integer produced by the multiplication just described is less than 2 ⁵³.
   51235 is not too large. But suppose that, in addition to 1.23, 5.11235, and 6.789, your range of X and Y coordinates includes a fourth decimal value, 10000000006.789876. Because this coordinate's decimal precision is longer than that of the other three, you would multiply this coordinate-not 5.1235-by a power of 10. To convert it to an integer, you could multiply it by 10 to the sixth power (1000000). But the resulting value, 10000000006789876, is greater than 2 ⁵³. If Db2® Spatial Extender tried to store it, the results would be unpredictable.

   To avoid this problem, select a power of 10 that, when multiplied by the original coordinate, yields a decimal number that Db2 Spatial Extender can truncate to a storable integer, with minimum loss of precision. In this case, you could select 10 to the fifth power (100000). Multiplying 100000 by 10000000006.789876 yields 1000000000678987.6. Db2 Spatial Extender would round this number to 1000000000678988, reducing its accuracy slightly.