Topic
  • 8 replies
  • Latest Post - ‏2013-09-09T00:09:24Z by aj_is_here
aj_is_here
aj_is_here
14 Posts

Pinned topic Similar exit function in Primitive operator

‏2013-09-02T12:01:42Z |

How to for terminate a operator or the whole job in streams.

  • Stan
    Stan
    76 Posts

    Re: Similar exit function in Primitive operator

    ‏2013-09-03T16:48:04Z  

    Please clarify the question particularly what is meant by"

    Similar exit funciton - similar to what?

    What do you want to happen when a operator is terminated?

    Are you looking to automate /  administer the applicaiton or job from with an operator?

     Perhaps describing a processing scenario with the applicaiton graph and operators then under what condition you would want to perform an EXIT and what would be Exited / Stopped / Cancelled. 

    How to cacel running jobs is documented here (canceljob/stopinstance) but I suspect this not what you mean by 'similar exit function'

    http://pic.dhe.ibm.com/infocenter/streams/v3r1/topic/com.ibm.swg.im.infosphere.streams.install-admin.doc/doc/ibminfospherestreams-stopping-apps-and-instances.html

     

     

  • aj_is_here
    aj_is_here
    14 Posts

    Re: Similar exit function in Primitive operator

    ‏2013-09-05T05:31:25Z  
    • Stan
    • ‏2013-09-03T16:48:04Z

    Please clarify the question particularly what is meant by"

    Similar exit funciton - similar to what?

    What do you want to happen when a operator is terminated?

    Are you looking to automate /  administer the applicaiton or job from with an operator?

     Perhaps describing a processing scenario with the applicaiton graph and operators then under what condition you would want to perform an EXIT and what would be Exited / Stopped / Cancelled. 

    How to cacel running jobs is documented here (canceljob/stopinstance) but I suspect this not what you mean by 'similar exit function'

    http://pic.dhe.ibm.com/infocenter/streams/v3r1/topic/com.ibm.swg.im.infosphere.streams.install-admin.doc/doc/ibminfospherestreams-stopping-apps-and-instances.html

     

     

    sorry for the incomplete details.

     

    What I meant is the exit function of C++

     

    exit(EXIT_FAILURE) or exit(1)

     

    wherein I want to gracefully terminate my application If I encountered some certain error.

    Thank You :D

  • Stan
    Stan
    76 Posts

    Re: Similar exit function in Primitive operator

    ‏2013-09-05T16:59:08Z  

    sorry for the incomplete details.

     

    What I meant is the exit function of C++

     

    exit(EXIT_FAILURE) or exit(1)

     

    wherein I want to gracefully terminate my application If I encountered some certain error.

    Thank You :D

    Thank you for clarfying that point.  I don't know of a way to do this but it may be possible to code such an exit procedure to spawn a kill thread that performs a canceljob and / or stop instance.  I will ask the SPL team to comment on this.

  • KanatT.
    KanatT.
    13 Posts

    Re: Similar exit function in Primitive operator

    ‏2013-09-05T20:51:48Z  
    • Stan
    • ‏2013-09-05T16:59:08Z

    Thank you for clarfying that point.  I don't know of a way to do this but it may be possible to code such an exit procedure to spawn a kill thread that performs a canceljob and / or stop instance.  I will ask the SPL team to comment on this.

    What behavior do you need from a graceful exit?  One idea is to use the C/C++'s exit function, which will cause the PE where your operator lives to fail---and separately, you write a script to monitor the health of the PEs, so you can cancel the job once you detect a dead PE.  Streams currently doesn't provide a mechanism from inside an operator to signal the other operators to wind down and eventually abort a job.

     

  • Kevin_Foster
    Kevin_Foster
    98 Posts

    Re: Similar exit function in Primitive operator

    ‏2013-09-07T23:02:58Z  
    • KanatT.
    • ‏2013-09-05T20:51:48Z

    What behavior do you need from a graceful exit?  One idea is to use the C/C++'s exit function, which will cause the PE where your operator lives to fail---and separately, you write a script to monitor the health of the PEs, so you can cancel the job once you detect a dead PE.  Streams currently doesn't provide a mechanism from inside an operator to signal the other operators to wind down and eventually abort a job.

     

    Or if you want to cancel the entire job, there are SPL functions that will return the instance name and job ID.

    http://pic.dhe.ibm.com/infocenter/streams/v3r1/topic/com.ibm.swg.im.infosphere.streams.spl-standard-toolkit-builtins.doc/spl-builtins.html?resultof=%22%63%65%69%6c%22%20

    I'd probably create an additional output port on my C++ or Java operator whose only purpose is to trigger the shutdown sequence in a downstream Custom operator. With those functions you could formulate a streamtool command and run that with the LinuxPipe operator.

    Would probably also be a good idea to create a log record entry as part of the shutdown sequence.

    -Kevin

  • aj_is_here
    aj_is_here
    14 Posts

    Re: Similar exit function in Primitive operator

    ‏2013-09-09T00:05:59Z  
    • KanatT.
    • ‏2013-09-05T20:51:48Z

    What behavior do you need from a graceful exit?  One idea is to use the C/C++'s exit function, which will cause the PE where your operator lives to fail---and separately, you write a script to monitor the health of the PEs, so you can cancel the job once you detect a dead PE.  Streams currently doesn't provide a mechanism from inside an operator to signal the other operators to wind down and eventually abort a job.

     

    Actually when I tried exit(EXIT_FAILURE) it shows the application is unhealthy, but the problem I have is I encounter data loss, when the PE before the element keeps on sending data, and I am unable to rollback the data because it keeps on running and processing. Although your suggestion is a good idea if I see the whole job as unhealthy I'll just cancel the job from a different application.

  • aj_is_here
    aj_is_here
    14 Posts

    Re: Similar exit function in Primitive operator

    ‏2013-09-09T00:08:33Z  

    Or if you want to cancel the entire job, there are SPL functions that will return the instance name and job ID.

    http://pic.dhe.ibm.com/infocenter/streams/v3r1/topic/com.ibm.swg.im.infosphere.streams.spl-standard-toolkit-builtins.doc/spl-builtins.html?resultof=%22%63%65%69%6c%22%20

    I'd probably create an additional output port on my C++ or Java operator whose only purpose is to trigger the shutdown sequence in a downstream Custom operator. With those functions you could formulate a streamtool command and run that with the LinuxPipe operator.

    Would probably also be a good idea to create a log record entry as part of the shutdown sequence.

    -Kevin

    Hi Sir Kevin,

     

    So if the PE becomes unhealthy ill just submit a data stream from the shutdown process function in a primitive operator? and receive from the custom operator to cancel the job.

     

    Thanks.

  • aj_is_here
    aj_is_here
    14 Posts

    Re: Similar exit function in Primitive operator

    ‏2013-09-09T00:09:24Z  

    Hi Sir Kevin,

     

    So if the PE becomes unhealthy ill just submit a data stream from the shutdown process function in a primitive operator? and receive from the custom operator to cancel the job.

     

    Thanks.

    In the custom operator, do you mean I should use the abort function? from the function list?