Topic
  • 2 replies
  • Latest Post - ‏2012-04-28T09:06:46Z by SuhaasKrishna
SuhaasKrishna
SuhaasKrishna
5 Posts

Pinned topic Compute Grid V8 Queries

‏2012-04-12T12:17:24Z |
Hi Sir,
I have folowing queires about usage of compute grid.Help me out.
1.Can I have zero inputStreams and zero outputStreams for a batch job step.i.e I want to use my own java input and output streams and own checkpoint maintanance etc not CG's.

2.How Compute Intensive job serves for above case.

3.Is there any way of parallel processing of **multiple batch job steps** which not shares common logic.(I Know parallel processing(PJM) using partitioning of single step which shares common logic)

4.Explain any case for which compute intensive job style really suits?

thanks
Suhaas
Updated on 2012-04-28T09:06:46Z at 2012-04-28T09:06:46Z by SuhaasKrishna
  • SystemAdmin
    SystemAdmin
    783 Posts

    Re: Compute Grid V8 Queries

    ‏2012-04-16T13:13:02Z  
    Suhaas,

    1.Can I have zero inputStreams and zero outputStreams for a batch job step.i.e I want to use my own java input and output streams and own checkpoint maintanance etc not CG's.

    Yes, you can have a batch step without batch data streams. I am curious: why do you want to do all this yourself? We believe WCG offers a flexible model for managing transactional batch and checkpoint/restart. If you choose to undertake these duties on your own, I recommend you use the transactional batch model (not Compute Intensive) and consider using the job step context persistent user data area for your checkpoint data.

    2.How Compute Intensive job serves for above case.

    Compute Invensive (CI) could be used for above case. But it is not a good fit. The difference is with CI you have to manage your own transactions with UserTransaction, manage your own data persistence, and CI does not have a "restartable" end state. CI has only 'ended' and 'failed' states. You can't restart a CI job. So CI is not a good option if you want to support checkpoint/restart.

    3.Is there any way of parallel processing of **multiple batch job steps** which not shares common logic.(I Know parallel processing(PJM) using partitioning of single step which shares common logic)

    No. All parallel subjobs are same logic with different parameters. We have been thinking about adding support for concurrent job steps (e.g. run step1 and step2 at same time). We would love if you would tell us more about your use case to help us with requirements.

    4.Explain any case for which compute intensive job style really suits?

    CI is good for anything that is not record processing - e.g. FTP, invoke shell script, invoke web service, etc.

    -Chris
  • SuhaasKrishna
    SuhaasKrishna
    5 Posts

    Re: Compute Grid V8 Queries

    ‏2012-04-28T09:06:46Z  
    Suhaas,

    1.Can I have zero inputStreams and zero outputStreams for a batch job step.i.e I want to use my own java input and output streams and own checkpoint maintanance etc not CG's.

    Yes, you can have a batch step without batch data streams. I am curious: why do you want to do all this yourself? We believe WCG offers a flexible model for managing transactional batch and checkpoint/restart. If you choose to undertake these duties on your own, I recommend you use the transactional batch model (not Compute Intensive) and consider using the job step context persistent user data area for your checkpoint data.

    2.How Compute Intensive job serves for above case.

    Compute Invensive (CI) could be used for above case. But it is not a good fit. The difference is with CI you have to manage your own transactions with UserTransaction, manage your own data persistence, and CI does not have a "restartable" end state. CI has only 'ended' and 'failed' states. You can't restart a CI job. So CI is not a good option if you want to support checkpoint/restart.

    3.Is there any way of parallel processing of **multiple batch job steps** which not shares common logic.(I Know parallel processing(PJM) using partitioning of single step which shares common logic)

    No. All parallel subjobs are same logic with different parameters. We have been thinking about adding support for concurrent job steps (e.g. run step1 and step2 at same time). We would love if you would tell us more about your use case to help us with requirements.

    4.Explain any case for which compute intensive job style really suits?

    CI is good for anything that is not record processing - e.g. FTP, invoke shell script, invoke web service, etc.

    -Chris
    Thank you very much Chris for your reply.

    As you asked me about my use case,following is the scenario. How to implement it with compute grid.

    Two applications app1 and app2, are regular j2ee applications(not batch) which perform some business logic and end up with storing some results in data base called MyDB.

    I have a batch application,single job consists of 3 job steps,first step is independent but step 2 is dependent on above mentioned app1 and app2 i.e step 2 should read data from MyDB after app1 and app2 work done. So batch application should wait for app1 and app2 to finish their work after executing step 1 of batch job.Once app1 and app2 finishes their works rest of the batch job should be completed normally.

    I tried this scenario by calling Thread.sleep(time) method form createjobstep() method of step 2 to wait for MyDB ready to process. It is working fine.Is there any problem with this approach in real time.

    Is compute Grid providing any cleaner way of dealing this scenario instead of using Thread.sleep().
    Thanks in Advance

    SuhaasKrishna