Topic
3 replies Latest Post - ‏2013-08-15T16:15:14Z by Petr Vilím
Elvamr
Elvamr
3 Posts
ACCEPTED ANSWER

Pinned topic What to expect of more workers on the objective

‏2013-07-09T15:19:09Z |

Hi all,

I'm building a scheduling model with OPL in IBM ILOG CPLEX Optimization Studio v12.5.0. We executed some tests and discovered that a higher number of workers does not have a positive influence on the objective. 2 workers is better than one, but 4 workers find worse solutions than 2 (but more solutions and it is close), 8, 16 and 32 workers do not come close with the same number of solutions.

Can anybody explain this behaviour?

Best regards,

Ellen

  • ol
    ol
    19 Posts
    ACCEPTED ANSWER

    Re: What to expect of more workers on the objective

    ‏2013-07-10T14:57:56Z  in response to Elvamr

    Hello,

    if the number of cores is 2, having more than 2 workers is likely to degrade the performances (the third worker needs to suspend worker1 or worker2 to be executed). Is it your case?

    In the case where the number of cores is greater than the number of workers, increasing the number of workers generally improves the efficiency: each worker runs a different search algorithm, and they all compete to find a solution. 

    Note that, even with more cores than workers, it may happen that (for some reason like memory contention, differences on algorithms, ...) increasing the number of workers does not help or even degrade the performances.

     

    Regards,

    Olivier

    • Elvamr
      Elvamr
      3 Posts
      ACCEPTED ANSWER

      Re: What to expect of more workers on the objective

      ‏2013-07-11T06:42:21Z  in response to ol

      Hi Olivier,

      Thanks for your explanation. We work with more cores (32) than workers.

      As I mentioned, for a small puzzle, 2 workers is better than one, but 4 workers find worse solutions than 2 (but more solutions and it is close). 8, 16 and 32 workers do not come close with the same number of solutions. As this is quite a small puzzle, memory contention shouldn't be the problem, but maybe the puzzle "is that simple" that more workers "confuse" each other.

      For a bigger puzzle, 8 workers seem to perform best, 1 and 2 workers don't even come close, 4 and 12 do come close. 12 workers find more solutions but not as good as the final solution of 8 workers. I think memory contention will be (one of) the cause(s) here.

      Regards,

      Ellen

      • Petr Vilím
        Petr Vilím
        5 Posts
        ACCEPTED ANSWER

        Re: What to expect of more workers on the objective

        ‏2013-08-15T16:15:14Z  in response to Elvamr

        Hello Ellen,

        note that to make a fair comparison, it is necessary to repeat the measurement several times with differen setting of RandomSeed parameter. The default search in CP Optimizer is randomized, so it may get sometimes lucky and sometimes unlucky. Additional workers are influencing the search and so a seed that is "lucky" for 2 workers may be "unlucky" for 4 or 8 workers.

        Also note that additional workers are not completely for free, even if there is enough CPU cores and enough memory. Initialization, synchronization and communication between workers also take some time.

        Best regards, Petr