Lack of memory

Documents CPLEX behavior in limited memory for LP models.

To sustain computational speed, CPLEX should use only available physical memory, rather than virtual memory or paged memory. Even if your problem data fit in memory, CPLEX will need still more memory to optimize the problem. When CPLEX recognizes that only limited memory is available, it automatically makes algorithmic adjustments to compensate. These adjustments almost always reduce optimization speed. If you detect when these automatic adjustments occur, then you can decide when you need to add additional memory to your computer to sustain computational speed for your particular problem.

An alternative to obtaining more memory is to conserve memory that is available. The memory emphasis parameter (memory reduction switch) can help you in this respect.

  • C++ Name MemoryEmphasis (bool) in Concert Technology

  • C Name CPX_PARAM_MEMORYEMPHASIS (int) in the Callable Library

  • emphasis memory in the Interactive Optimizer

If you set the memory emphasis parameter to its optional value of 1 (one), then CPLEX adopts memory conservation tactics at the beginning of optimization rather than only after the shortage becomes apparent. These tactics may still have a noticeable impact on solution speed because these tactics change the emphasis from speed to memory utilization, but they could give an improvement over the default in the case where memory is insufficient.

The following sections offer further guidance about memory conservation if memory emphasis alone does not do enough for your problem.

Warning messages

In certain cases, CPLEX issues a warning message when it cannot perform an operation, but it continues to work on the problem. Other CPLEX messages warn that CPLEX is compressing the problem to conserve memory. These warnings mean that CPLEX finds insufficient memory available, so it is following an alternate—less desirable—path to a solution. If you provide more memory, CPLEX will return to the best path toward a solution.

Paging virtual memory

If you observe paging of memory to disk, then your application is incurring a performance penalty. If you increase available memory in such a case, performance will speed up dramatically.

Refactoring frequency and memory requirements

The CPLEX primal and dual simplex optimizers refactor the problem basis at a rate set by the ReInv parameter.

The longer CPLEX works between refactoring, the greater the amount of memory it needs for each iteration. Consequently, one way of conserving memory is to decrease the interval between refactoring. In fact, if little memory is available to it, CPLEX will automatically decrease the refactoring interval in order to use less memory at each iteration.

Since refactoring is an expensive operation, decreasing the refactoring interval (that is, factoring more often) will generally slow performance. You can tell whether performance is being degraded in this way by checking the iteration log file.

In an extreme case, lack of memory may force CPLEX to refactor at every iteration, and the impact on performance will be dramatic. If you provide more memory in such a situation, the benefit will be tremendous.

Preprocessing and memory requirements

By default, CPLEX automatically preprocesses your problem before optimizing, and this preprocessing requires memory. If memory is extremely tight, consider turning off preprocessing, by setting the PreInd parameter to 0 (zero). But doing this foregoes the potential performance benefit of preprocessing, and should be considered only as a last resort.