I'm new to Streams and I am trying to find out how the number of processing element containers (PEC) is defined in Streams.
Or is it totally dynamic, entirely controlled by Streams, spawning additional PECs as needed?
Is there a pointer to where in the documentation one can find a description on the configuration and life cycle of PECs?
Sorry if this information is in a rather obvious place and I couldn't find it.
Thanks for your help.
This topic has been locked.
3 replies Latest Post - 2012-10-05T21:10:20Z by jlerm
Pinned topic Defining the number of PECs
Answered question This question has been answered.
Unanswered question This question has not been answered yet.
SystemAdmin 110000D4XK1245 PostsACCEPTED ANSWER
Re: Defining the number of PECs2012-10-05T05:23:40Z in response to jlermHi Julias - you have quite a bit if control over operator placement and fusion within a PE. See the operator config guide as a start
mendell 120000KPGC219 PostsACCEPTED ANSWER
Re: Defining the number of PECs2012-10-05T12:22:49Z in response to jlermThere is one PEC for each PE (processing element) in the SPL program. By default, the SPL compiler puts each operator into its own PE, but you have control over that using config placement directives in your SPL program. You can also do a 'profiling run', and then the SPL compiler will use the results of that run to try to pick an optimal PE placement, minimizing data transfer time, while ensuring that CPU use isn't too high. You can then repeat as necessary.
PECs (and the associated PEs) are created a job submission time, and remain active until the job is cancelled.