Topic
IC4NOTICE: developerWorks Community will be offline May 29-30, 2015 while we upgrade to the latest version of IBM Connections. For more information, read our upgrade FAQ.
3 replies Latest Post - ‏2012-10-05T21:10:20Z by jlerm
jlerm
jlerm
3 Posts
ACCEPTED ANSWER

Pinned topic Defining the number of PECs

‏2012-10-05T01:27:46Z |
I'm new to Streams and I am trying to find out how the number of processing element containers (PEC) is defined in Streams.
Or is it totally dynamic, entirely controlled by Streams, spawning additional PECs as needed?
Is there a pointer to where in the documentation one can find a description on the configuration and life cycle of PECs?
Sorry if this information is in a rather obvious place and I couldn't find it.
Thanks for your help.

Julius
  • SystemAdmin
    SystemAdmin
    1245 Posts
    ACCEPTED ANSWER

    Re: Defining the number of PECs

    ‏2012-10-05T05:23:40Z  in response to jlerm
    Hi Julias - you have quite a bit if control over operator placement and fusion within a PE. See the operator config guide as a start

    http://publib.boulder.ibm.com/infocenter/streams/v2r0/index.jsp?nav=%2F2_1_3

    Thanks
  • mendell
    mendell
    219 Posts
    ACCEPTED ANSWER

    Re: Defining the number of PECs

    ‏2012-10-05T12:22:49Z  in response to jlerm
    There is one PEC for each PE (processing element) in the SPL program. By default, the SPL compiler puts each operator into its own PE, but you have control over that using config placement directives in your SPL program. You can also do a 'profiling run', and then the SPL compiler will use the results of that run to try to pick an optimal PE placement, minimizing data transfer time, while ensuring that CPU use isn't too high. You can then repeat as necessary.

    PECs (and the associated PEs) are created a job submission time, and remain active until the job is cancelled.

    Mark
  • jlerm
    jlerm
    3 Posts
    ACCEPTED ANSWER

    Re: Defining the number of PECs

    ‏2012-10-05T21:10:20Z  in response to jlerm
    Thanks a lot for your replies!