Topic
IC4NOTICE: developerWorks Community will be offline May 29-30, 2015 while we upgrade to the latest version of IBM Connections. For more information, read our upgrade FAQ.
4 replies Latest Post - ‏2013-01-21T07:58:14Z by hllsen
hllsen
hllsen
57 Posts
ACCEPTED ANSWER

Pinned topic Best coding practice for IloEnv of the subproblems in delayed generation

‏2013-01-18T13:14:06Z |
Hi,

In order to use it in a constraint generation process, I need to solve a subproblem in ILOLAZYCONSTRAINTCALLBACKX. What I wonder is which one is better coding practice among the following and why if it's not a hassle to explain:

Separate environment for master and subproblems:
1- Subproblem env created and deleted in the callback.
2- Subproblem env created before cplex_master.solve(), model and cplex objects of subproblems created and deleted in the callback.
3- Subproblem env, model, and cplex objects initialized before cplex_master.solve(), and modified as needed in the callbacks.

Using the same IloEnv object for master and sub-problems:
4- Model and cplex objects of subproblems created before hand and modified in the callback.

I know the decision somehow project and problem specific. What I am trying to achieve is the following: the only difference between any two subproblems is in the objective function coefficients; therefore, I wonder which option should I use?

Cheers,
h.
Updated on 2013-01-21T07:58:14Z at 2013-01-21T07:58:14Z by hllsen
  • SystemAdmin
    SystemAdmin
    7929 Posts
    ACCEPTED ANSWER

    Re: Best coding practice for IloEnv of the subproblems in delayed generation

    ‏2013-01-18T14:44:57Z  in response to hllsen
    If you can afford the additional overhead then I would go for a new environment/model in each callback invocation. That is the safest and cleanest thing to do, in particular when it comes to multi-threading. With this approach environment and model are always local to the callback and you need to bother whether callbacks are invoked in parallel etc. Moreover, you can do a subenv.end() at the end of the callback end be sure that everything is cleaned up properly.
    If the induced overhead is not acceptable then I would first check if there is a way to create the model faster (for example by just cloning global data).
    I am not exactly sure at the moment but I think using the same environment for solving master and sub-problem at the same time may not even be officially supported. So the alternative to the callback-local environment is an additional global environment that is created before the solve is started.
    • hllsen
      hllsen
      57 Posts
      ACCEPTED ANSWER

      Re: Best coding practice for IloEnv of the subproblems in delayed generation

      ‏2013-01-19T12:09:16Z  in response to SystemAdmin
      Thank you Daniel. I have two follow up questions:

      If you mean only "the time to create new objects in every callback" by "the additional overhead", then it is not a problem since "Build_Time"s are relatively small. As I was reading my question again, I realized that I couldn't clearly explain myself. My concern is to lose the useful information (variable values, optimal basis, duals, etc..) that can be used in the (re)optimization of next sub-problem and shorten the time (some of the information is going to be valid since feasible space is the same for all sub-problems (i.e. only objective coefficients change)). So, does it (keeping these info) shortens the optimization time if the only difference between two problems is the objective coefficients?

      I am afraid you are right about the tidiness of the "new sub-problem environment" approach, however I think I have a solution to the problems related to parallelism of sub-problem environments. Since Cplex gives each thread an id number, if we create #_of_processors number of sub-problem environments (before cplex.solve()) then each callback calls the sub-problem that is indexed with the cplex_thread_id. Therefore, each sub-problem environment can be simultaneously accessed from only one thread and information of the last solution can be used in the re-optimization (and my programming knowledge is not enough :( to understand whether this makes sense or not?).
      • SystemAdmin
        SystemAdmin
        7929 Posts
        ACCEPTED ANSWER

        Re: Best coding practice for IloEnv of the subproblems in delayed generation

        ‏2013-01-21T07:31:19Z  in response to hllsen
        > hllsen wrote:
        > Thank you Daniel. I have two follow up questions:
        >
        > If you mean only "the time to create new objects in every callback" by "the additional overhead", then it is not a problem since "Build_Time"s are relatively small. As I was reading my question again, I realized that I couldn't clearly explain myself. My concern is to lose the useful information (variable values, optimal basis, duals, etc..) that can be used in the (re)optimization of next sub-problem and shorten the time (some of the information is going to be valid since feasible space is the same for all sub-problems (i.e. only objective coefficients change)). So, does it (keeping these info) shortens the optimization time if the only difference between two problems is the objective coefficients?
        >
        OK, I see. In this case re-creating a new environment in each callback invocation will result in losing any sort of warmstart information. You could of course store the warmstart information in some global data and feed it back to the solver after re-creating the environment. But that is essentially the same as working with a global environment :-)
        So I would go for what you describe below.
        > I am afraid you are right about the tidiness of the "new sub-problem environment" approach, however I think I have a solution to the problems related to parallelism of sub-problem environments. Since Cplex gives each thread an id number, if we create #_of_processors number of sub-problem environments (before cplex.solve()) then each callback calls the sub-problem that is indexed with the cplex_thread_id. Therefore, each sub-problem environment can be simultaneously accessed from only one thread and information of the last solution can be used in the re-optimization (and my programming knowledge is not enough :( to understand whether this makes sense or not?).
        >
        Yes, that makes a lot of sense and I think in your case is one way to go.
        Another (more elegant IMO) way is to not use the ILOLAZYCONSTRAINTCALLBACKX() macro. If you don't use that macro but implement your class manually then you will notice that you have to implement a function duplicateCallback(). This function is used to duplicate the callback for each thread that CPLEX uses. The trick is to not copy the thread-local environment here but to instead create a new thread local environment for each thread (untested):
        
        
        
        class LazyConstraintCallback : 
        
        public IloCplex::LazyConstraintCallbackI 
        { 
        // Your callback members here ... IloEnv threadEnv;     
        // Thread local environment for subproblems. IloModel threadModel; 
        // Thread local model IloCplex threadCplex; 
        // Thread local solver 
        
        public: LazyConstraintCallback(
        /* initializers for your callback members */) : 
        // Initialize your callback members. ... 
        // and create a _new_ environment, model, solver for each thread threadEnv(), threadModel(threadEnv), threadCplex(threadModel) 
        { 
        } ~LazyConstraintCallback() 
        { threadEnv.end(); 
        } CallbackI *duplicateCallback() 
        
        const 
        { 
        
        return 
        
        new (getEnv()) LazyConstraintCallback(
        /* initializers for your callback members */); 
        } 
        
        void main() 
        { 
        // separation code, use threadModel and threadCplex to solve the subproblem. 
        } 
        };
        
        • hllsen
          hllsen
          57 Posts
          ACCEPTED ANSWER

          Re: Best coding practice for IloEnv of the subproblems in delayed generation

          ‏2013-01-21T07:58:14Z  in response to SystemAdmin
          Again, thank you Daniel for both the code snippet and the explanations. I agree with you on that using vector's to keep the thread information separate is inelegant and dirty but as you already said (I also think the same) it's my only way :). I'll definitely try to use the method you proposed, in the future though.

          Cheers,
          h.