• 3 replies
  • Latest Post - ‏2012-10-05T21:10:20Z by jlerm
3 Posts

Pinned topic Defining the number of PECs

‏2012-10-05T01:27:46Z |
I'm new to Streams and I am trying to find out how the number of processing element containers (PEC) is defined in Streams.
Or is it totally dynamic, entirely controlled by Streams, spawning additional PECs as needed?
Is there a pointer to where in the documentation one can find a description on the configuration and life cycle of PECs?
Sorry if this information is in a rather obvious place and I couldn't find it.
Thanks for your help.

  • SystemAdmin
    1245 Posts

    Re: Defining the number of PECs

    Hi Julias - you have quite a bit if control over operator placement and fusion within a PE. See the operator config guide as a start

  • mendell
    219 Posts

    Re: Defining the number of PECs

    There is one PEC for each PE (processing element) in the SPL program. By default, the SPL compiler puts each operator into its own PE, but you have control over that using config placement directives in your SPL program. You can also do a 'profiling run', and then the SPL compiler will use the results of that run to try to pick an optimal PE placement, minimizing data transfer time, while ensuring that CPU use isn't too high. You can then repeat as necessary.

    PECs (and the associated PEs) are created a job submission time, and remain active until the job is cancelled.

  • jlerm
    3 Posts

    Re: Defining the number of PECs

    Thanks a lot for your replies!