Topic
7 replies Latest Post - ‏2013-08-02T05:27:04Z by DanielJunglas
FANS_Nader_Al_Theeb
FANS_Nader_Al_Theeb
3 Posts
ACCEPTED ANSWER

Pinned topic Clear Models and Memory leak

‏2013-06-29T21:47:40Z |

Dear all,

I discussed this topic in the past, but I have some issues. Let's see the following code

--------------------------------------------------------------------------------------------------

Create model // name = model

for i=1:100

IloCplex cplex(env);
cplex.extract(model);

IloModel model2(env);
model2.add(model);
cplex.extract(model2);

modify the model2

solve and save the objective value

        cplex.clearModel();
        cplex.end();
        model2.end();

end for

---------------------------------------------------------------------------------------

 

My problem is that even I clear the model at the end of each iteration, a small portion of memory is used somewhere and not cleared. This cause the iterations to take longer time (5 seconds per iteration for the first 10 iterations, 30 seconds per iteration for the last 10 iterations)

Anyone knows why this happened and where the memory is lost?

Thanks

 

 

 

  • DanielJunglas
    DanielJunglas
    110 Posts
    ACCEPTED ANSWER

    Re: Clear Models and Memory leak

    ‏2013-07-01T05:08:08Z  in response to FANS_Nader_Al_Theeb

    What version of CPLEX do you use? How do you assess that memory is leaked? Do you have a tool that actually reports the leakage?

    How exactly do you modify model2? Maybe the code that performs the modification leaks memory?

    Could you please print out env.getMemoryUsage() at the beginning of each iteration and check whether this value increases?

    How large is the amount of memory that is leaked? I have a hard time believing that a small memory leak will result in a 25 second slowdown. Do you apply the exact same modifications in each iteration or may the slowdown be due to the fact that you are solving different models?

    • FANS_Nader_Al_Theeb
      FANS_Nader_Al_Theeb
      3 Posts
      ACCEPTED ANSWER

      Re: Clear Models and Memory leak

      ‏2013-07-01T17:44:00Z  in response to DanielJunglas


      Dear Daniel
      Thanks for your replay

      I'm using CPLEX-Concert 12.2 in Linux machine, I was observing the memory usage in other terminal using top command. Now, I'm using env.getMemoryUsage() as you suggest, I got the following results (iteration number        memory usage)

      ------------------------------------------------------

      0         memory usage =  394496256
      1         memory usage =  826231320
      2         memory usage =  1266370088
      3         memory usage =  1672823352
      4         memory usage =  2146386416
      5         memory usage =  2552839968
      6         memory usage =  2959490992
      7         memory usage =  3365945552
      8         memory usage =  3906879480
      9         memory usage =  4313135200
      10         memory usage =  4719785576
      11         memory usage =  5126437104
      12         memory usage =  5532693328

      ------------------------------------------------------

      I modify model 2 by enforcing some variables to take a value and keep others free, this modifications are the same in all iterations.

      Is there anything else can be done to solve this problem? For example, Do I need to clear variables or change arrangement of extract, clear, and end commands in my code?

      Thanks
       

      • Lessi
        Lessi
        5 Posts
        ACCEPTED ANSWER

        Re: Clear Models and Memory leak

        ‏2013-07-31T20:10:38Z  in response to FANS_Nader_Al_Theeb

        I usually clear variables, constraints, etc.

         

      • DanielJunglas
        DanielJunglas
        110 Posts
        ACCEPTED ANSWER

        Re: Clear Models and Memory leak

        ‏2013-08-01T06:02:53Z  in response to FANS_Nader_Al_Theeb

        Sorry for the late reply. I missed your answer :-(

        It indeed looks like you or CPLEX (or both) are leaking memory here. Does the problem persist if you skip the "modify model2" step in the loop (that, solve the same model in each loop)?

        • FANS_Nader_Al_Theeb
          FANS_Nader_Al_Theeb
          3 Posts
          ACCEPTED ANSWER

          Re: Clear Models and Memory leak

          ‏2013-08-01T07:28:58Z  in response to DanielJunglas

          Thanks all for replays

          I found what is the problem,

          originally I use  (IloEnv env;) before the for loop and (env.end();) after the for loop

          Now, I use (IloEnv env;) at the beginning of each iteration (inside for loop) and (env.end();) before the end of for loop

          I did this by chance and I don't know what the idea behind this, if you would explain why

          Thanks
          Nader
           

          • PaulRubin
            PaulRubin
            41 Posts
            ACCEPTED ANSWER

            Re: Clear Models and Memory leak

            ‏2013-08-01T19:21:44Z  in response to FANS_Nader_Al_Theeb

            Some operations can produce unintended persistent byproducts. I think += does this, although I'm not positive about that. Since C++ does not have automatic garbage collection, these byproducts can accumulate (and consume memory). Ending the environment to which they belong clears them out. I don't know if your code produced any byproducts, but it is one possible explanation.

            Paul

            • DanielJunglas
              DanielJunglas
              110 Posts
              ACCEPTED ANSWER

              Re: Clear Models and Memory leak

              ‏2013-08-02T05:27:04Z  in response to PaulRubin

              Paul,

              it is usually the other way around. In C++-Concert a statement like 'expr += ...' will usually avoid the creation of "byproducts" while something like 'expr = expr + ...' will leak memory.

              Nader,

              if calling env.end() fixes your problem then almost certainly the old code was missing appropriate end() statements in some places. Like Paul said, it is not always blatantly obvious on which objects on has to call end() :-(