Memory limit
Calculating the correlations between categorical fields takes a lot of memory. There is a memory limit to avoid that the mining run locks up.
If the memory limit is exceeded, the calculation of as many correlations as required to meet this limit is stopped. This might result in slightly worse results, however, the mining process completes successfully.
If the memory limit is too low, you are notified. Repeating the mining run with a higher limit ensures the optimum result.
You can change the memory limit by using the
power option '-buf'.
Another
factor is the number of different values that are tracked for each
field. By default, 100 different values are distinguished and taken
into account for correlation calculation. This usually ensures good
results. However, you can change this value by using the power option -IDM_MAX_DISCR_COUNT.
- If you raise the value for the memory limit, the results might improve slightly. However, this can have a dramatic impact on the memory consumption and performance, and this might lead to discarding of correlations.
- If you lower the value for the memory limit, the results of the correlation calculation can become unreliable.