IC5Notice: We have upgraded developerWorks Community to the latest version of IBM Connections. For more information, read our upgrade FAQ.
Topic
  • 5 replies
  • Latest Post - ‏2008-09-29T18:48:12Z by lcham
astdenis
astdenis
11 Posts

Pinned topic Loadleveler co-scheduling and max_idle.

‏2008-09-18T13:07:00Z |
In order to control queue flooding, we have set max_idle to 3 and it's been doing what we expect. Lately, our users have been puzzled by this behaviour: a job with 4+ co-scheduled steps will sit in the queue for ever...

Since co-scheduled steps must run concurrently, shouldn't they count as 1 in the idle queue?

Food for a PMR maybe?

Alain
Updated on 2008-09-29T18:48:12Z at 2008-09-29T18:48:12Z by lcham
  • lcham
    lcham
    7 Posts

    Re: Loadleveler co-scheduling and max_idle.

    ‏2008-09-23T15:04:43Z  
    Hi.
    Coscheduled steps are counted as individual idle steps.

    In the doc the following is stated,
    "All steps within a job with the coschedule keyword set to true will be coscheduled.
    The coscheduled steps will continue to be stored as individual steps in both memory and in the job queue, but when performing certain operations, such as scheduling, the steps will be managed as a single entity."

    Regards,
    Linda
  • astdenis
    astdenis
    11 Posts

    Re: Loadleveler co-scheduling and max_idle.

    ‏2008-09-25T11:48:20Z  
    • lcham
    • ‏2008-09-23T15:04:43Z
    Hi.
    Coscheduled steps are counted as individual idle steps.

    In the doc the following is stated,
    "All steps within a job with the coschedule keyword set to true will be coscheduled.
    The coscheduled steps will continue to be stored as individual steps in both memory and in the job queue, but when performing certain operations, such as scheduling, the steps will be managed as a single entity."

    Regards,
    Linda
    So how will co-scheduled job steps ever be managed as a single entity for scheduling if they never make it to the idle queue?

    Alain
  • lcham
    lcham
    7 Posts

    Re: Loadleveler co-scheduling and max_idle.

    ‏2008-09-25T14:32:54Z  
    • astdenis
    • ‏2008-09-25T11:48:20Z
    So how will co-scheduled job steps ever be managed as a single entity for scheduling if they never make it to the idle queue?

    Alain
    Coscheduled steps are treated as individual steps on the job queue so that limits such as max_idle can not be circumvented by simply defining multiple steps as coscheduled.

    Regards,
    Linda
  • astdenis
    astdenis
    11 Posts

    Re: Loadleveler co-scheduling and max_idle.

    ‏2008-09-26T20:57:44Z  
    • lcham
    • ‏2008-09-25T14:32:54Z
    Coscheduled steps are treated as individual steps on the job queue so that limits such as max_idle can not be circumvented by simply defining multiple steps as coscheduled.

    Regards,
    Linda
    Say I build a x step job with the intention of circumventing the max_idle limit. Won't the very fact I use co-scheduling defeat my purpose? My understanding of the max_idle limit is it's used to prevent queue flooding. If I fill the idle queue with x steps that will be scheduled at the same time, it means all the resources required by my job need to be available at that time, just like if it was a large single step job.

    May be I don't have enough imagination to see how one could abuse a system where co-scheduled steps are treated as one. Can you provide examples?

    Thanks.
    Alain.
  • lcham
    lcham
    7 Posts

    Re: Loadleveler co-scheduling and max_idle.

    ‏2008-09-29T18:48:12Z  
    • astdenis
    • ‏2008-09-26T20:57:44Z
    Say I build a x step job with the intention of circumventing the max_idle limit. Won't the very fact I use co-scheduling defeat my purpose? My understanding of the max_idle limit is it's used to prevent queue flooding. If I fill the idle queue with x steps that will be scheduled at the same time, it means all the resources required by my job need to be available at that time, just like if it was a large single step job.

    May be I don't have enough imagination to see how one could abuse a system where co-scheduled steps are treated as one. Can you provide examples?

    Thanks.
    Alain.
    Hi Alain,

    A possible scenario where coscheduled steps can be abused would be a queue with 5 job steps and max idle per user set to 1 where user A has 4 job steps which each require 25% of the cluster resources and user B has a job step which requires 75% of the cluster resources. Assuming the steps all have the same priority then without coschedule, steps A1 and B1 would run first. If A's job steps are changed to coschedule then all of A's job steps may run before B's job step.

    But we do see your point, by trying to protect against abuses of coschedule steps, we force the legitimate users of coschedule steps to increase queue limits thus allowing non coschedule steps to flood the queue. We don't think we can treat coscheduled steps as a single idle step without adding some administrative control over the use of coshedule steps.

    Regards,
    Linda