Topic
IC4NOTICE: developerWorks Community will be offline May 29-30, 2015 while we upgrade to the latest version of IBM Connections. For more information, read our upgrade FAQ.
23 replies Latest Post - ‏2013-02-25T08:57:24Z by SystemAdmin
SystemAdmin
SystemAdmin
1883 Posts
ACCEPTED ANSWER

Pinned topic Mem issue, each IloOplModel takes 13 MB

‏2013-02-12T15:00:38Z |
Hi

I'm implementing a column geneartion based algorithm as described in:

https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14939755&#14939755

As described this requires that i have a large number of IloOplModel objects existing at the same time, which gives me a speed up, but it really eats memory. The flow for creating the objects is as:
var subOplArrays = new Array(numVessels);
var subCplexArrays = new Array(numVessels);

...

for(var ves in MasterDataElmts.Vessels) {
subCplexArrays[i] = new IloCplex();
subOplArrays[i] = new IloOplModel(subDef, subCplexArrays[i]);

...
subOplArrays[i].addDataSource(subData);
subOplArrays[i].generate();
subOplArrays[i].settings.mainEndEnabled = true;
subCplexArrays[i].epgap = subProblemGap;
MasterDataElmts.bunkerPlanUniqueId = MasterDataElmts.bunkerPlanUniqueId +1;
writeln("Model object created for vessel: "+i);
i++;
}

this setup restricts me to have at most 100 subproblems due to mem issues, as each model takes about 13 MB. And i'd really like to go much higher.

So my question, what can i do to reduce memery load ? I've tried to make sure that subData only holds the absolutely necessary, and i cant reduce the model it self more than ive done.

Looking forward to any hints!

Christian
Updated on 2013-02-25T08:57:24Z at 2013-02-25T08:57:24Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    1883 Posts
    ACCEPTED ANSWER

    Re: Mem issue, each IloOplModel takes 13 MB

    ‏2013-02-12T19:24:03Z  in response to SystemAdmin
    Why don't you reuse the same sub model objects?
    • SystemAdmin
      SystemAdmin
      1883 Posts
      ACCEPTED ANSWER

      Re: Mem issue, each IloOplModel takes 13 MB

      ‏2013-02-13T06:56:58Z  in response to SystemAdmin
      subData is different for each sub problem and i would have to recreate the whole matrix, objective and rhs. So it would be better to destroy and recreate.
      • SystemAdmin
        SystemAdmin
        1883 Posts
        ACCEPTED ANSWER

        Re: Mem issue, each IloOplModel takes 13 MB

        ‏2013-02-13T09:40:01Z  in response to SystemAdmin
        You can call generate() for each sub problem instance (on the same model object), and only update the input data... I think I don't understand what is your problem, could you please clarify
        • SystemAdmin
          SystemAdmin
          1883 Posts
          ACCEPTED ANSWER

          Re: Mem issue, each IloOplModel takes 13 MB

          ‏2013-02-13T10:11:32Z  in response to SystemAdmin
          Sorry i'll elaborate some more.

          Originally i did as you suggested, calling generate for each subproblem in each iteration.

          But with this approach the "generate()" call ends up using 90 % or more of the computational time, as each of my subproblems are usually easily solved. So the whole idea of this different approach is to avoid the generate calls, by simply updating the changed dual values in the subproblems, as all other values are the same from iteration to iteration.

          The problem is now that all these subproblems eat all the mem, so it does scale to sufficiently large instances.
          • SystemAdmin
            SystemAdmin
            1883 Posts
            ACCEPTED ANSWER

            Re: Mem issue, each IloOplModel takes 13 MB

            ‏2013-02-13T11:11:17Z  in response to SystemAdmin
            In the code above I see a call to sub problem generate() inside the loop, so how did you reduce the number of calls there?
            • SystemAdmin
              SystemAdmin
              1883 Posts
              ACCEPTED ANSWER

              Re: Mem issue, each IloOplModel takes 13 MB

              ‏2013-02-13T12:02:13Z  in response to SystemAdmin
              As stated, i've only posted the part of the code creating the object. If you looked at the post of above link you'll get an idea of what the modyfing and solving code looks like.

              Cheers, Chrsitian