Topic
  • 3 replies
  • Latest Post - ‏2012-02-28T15:33:21Z by SystemAdmin
JohnnyPawan
JohnnyPawan
3 Posts

Pinned topic Paralle Job Manger API

‏2012-02-22T17:24:38Z |
Hi Sir,
This is my first thread regarding Compute Grid,I am working on Compute Grid v8, i have following queries regarding Parallel Job Manger API

1.Implementing Parameterizer API is mandatory and Are all other api namely synchronisation api ,subjob collector api, subjob analyzer api optional for *any Parallel Job Processing*? when other three are mandatory?
2.If we want to implement above three api, what logic we can put in its extended classes?In what way we get benefit by extending these abstract classes? Could you explain me in detail the purpose of each of these api?
3.How the check point mechanism applicable for the parallel jobs?Is top level Job checkpoint algorithm is applicable to parallel sub Jobs?
4.I am developing application using RAD, writing "run" property in xjcl is giving error
why?I tried with PJM sample application provided by IBM

Any other valid information related to PJM is appreciated

Thanks in advance
Updated on 2012-02-28T15:33:21Z at 2012-02-28T15:33:21Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    783 Posts

    Re: Paralle Job Manger API

    ‏2012-02-22T19:26:04Z  
    Dear User,

    Answers to your questions:

    1.Implementing Parameterizer API is mandatory and Are all other api namely synchronisation api ,subjob collector api, subjob analyzer api optional for *any Parallel Job Processing*? when other three are mandatory?

    Parameterizer is mandatory. The others are optional.

    2.If we want to implement above three api, what logic we can put in its extended classes? In what way we get benefit by extending these abstract classes? Could you explain me in detail the purpose of each of these api?

    1. Use required Parameterizer API to decide number of subjobs and unique substitution properties for each. Parameterizer runs on "top level job" job step thread.
    2. Use optional Synchronization API to have "logical transaction" demarcation across all subjobs. Synchronization runs on "top level job" job step thread. It has begin/commit/rollback callbacks that are invoked before subjobs are submitted and after subjobs complete. It is a logical demarcation of work, it is not a JTA transaction. The callbacks are for flow control purposes only, such as executing compensation logic - no resources are enlisted in a "logical transaction".
    3. Use optional SubJobCollector API to send application data (e.g. statistics) from subjobs to top-level-job. SubJobCollector runs on the job step thread of each subjob. It is invoked after each checkpoint interval and at the end of the subjob. The API can send an externalizable object to the SubJobAnalyzer (see below).
    4. Use optional SubJobAnalyzer API to receive application data from the SubJobCollector API. SubJobAnalyzer runs on "top level job" job step thread. It is invoked each time a SubJobCollector sends data (see above). It is also invoked with a subjob's return code, each time a subjob ends. The SubJobAnalyzer receives control for subjob end (to receive return code) whether or not the SubJobCollector is in use.

    3.How the check point mechanism applicable for the parallel jobs?Is top level Job checkpoint algorithm is applicable to parallel sub Jobs?

    Each subjob manages its own checkpoints. If your parallel job rolls back (i.e. ends in restartable state) it is up to the application to execute any necessary compensation logic - usually through the Synchronization API.

    4.I am developing application using RAD, writing "run" property in xjcl is giving error
    why?I tried with PJM sample application provided by IBM


    RAD 8 supports the WAS 8 Batch support. It does not yet support WCG 8. Note: WCG 8 is a superset of the batch function in WAS 8. You can get access to WCG 8 APIs in your RAD 8 batch projects by putting the WCG 8 com.ibm.ws.batch.runtime.jar file at the top of your project's build path. However, you cannot use WCG 8 xJCL extensions in the RAD xJCL editor. You have to do that outside of RAD. There should be a RAD update later this year to address this shortfall.
  • JohnnyPawan
    JohnnyPawan
    3 Posts

    Re: Paralle Job Manger API

    ‏2012-02-23T06:12:22Z  
    Dear User,

    Answers to your questions:

    1.Implementing Parameterizer API is mandatory and Are all other api namely synchronisation api ,subjob collector api, subjob analyzer api optional for *any Parallel Job Processing*? when other three are mandatory?

    Parameterizer is mandatory. The others are optional.

    2.If we want to implement above three api, what logic we can put in its extended classes? In what way we get benefit by extending these abstract classes? Could you explain me in detail the purpose of each of these api?

    1. Use required Parameterizer API to decide number of subjobs and unique substitution properties for each. Parameterizer runs on "top level job" job step thread.
    2. Use optional Synchronization API to have "logical transaction" demarcation across all subjobs. Synchronization runs on "top level job" job step thread. It has begin/commit/rollback callbacks that are invoked before subjobs are submitted and after subjobs complete. It is a logical demarcation of work, it is not a JTA transaction. The callbacks are for flow control purposes only, such as executing compensation logic - no resources are enlisted in a "logical transaction".
    3. Use optional SubJobCollector API to send application data (e.g. statistics) from subjobs to top-level-job. SubJobCollector runs on the job step thread of each subjob. It is invoked after each checkpoint interval and at the end of the subjob. The API can send an externalizable object to the SubJobAnalyzer (see below).
    4. Use optional SubJobAnalyzer API to receive application data from the SubJobCollector API. SubJobAnalyzer runs on "top level job" job step thread. It is invoked each time a SubJobCollector sends data (see above). It is also invoked with a subjob's return code, each time a subjob ends. The SubJobAnalyzer receives control for subjob end (to receive return code) whether or not the SubJobCollector is in use.

    3.How the check point mechanism applicable for the parallel jobs?Is top level Job checkpoint algorithm is applicable to parallel sub Jobs?

    Each subjob manages its own checkpoints. If your parallel job rolls back (i.e. ends in restartable state) it is up to the application to execute any necessary compensation logic - usually through the Synchronization API.

    4.I am developing application using RAD, writing "run" property in xjcl is giving error
    why?I tried with PJM sample application provided by IBM


    RAD 8 supports the WAS 8 Batch support. It does not yet support WCG 8. Note: WCG 8 is a superset of the batch function in WAS 8. You can get access to WCG 8 APIs in your RAD 8 batch projects by putting the WCG 8 com.ibm.ws.batch.runtime.jar file at the top of your project's build path. However, you cannot use WCG 8 xJCL extensions in the RAD xJCL editor. You have to do that outside of RAD. There should be a RAD update later this year to address this shortfall.
    Thanks alot Chris for your fast Response and answers to my questions.
    Could you upload new batchpackager to work in eclipse environment and WCG 8 and kindly resolve my following quires as well.

    1.I know the creation of Ear using batch packager but how to create xjcl using job properties file using batch packager
    2.can we develop any kind and any size of batch application using pojo model using eclipse and batch packager? which is the best way, using RAD or Eclipse with batch Packager in production use? what are pros and cons in both approaches?
    3.which is the best way,BDS generic batch step(implementing BatchRecordProcessor) or custom batch step(implementing batchstepInterface) for using single input and single output streams?

    Thanks in advance
  • SystemAdmin
    SystemAdmin
    783 Posts

    Re: Paralle Job Manger API

    ‏2012-02-28T15:33:21Z  
    Thanks alot Chris for your fast Response and answers to my questions.
    Could you upload new batchpackager to work in eclipse environment and WCG 8 and kindly resolve my following quires as well.

    1.I know the creation of Ear using batch packager but how to create xjcl using job properties file using batch packager
    2.can we develop any kind and any size of batch application using pojo model using eclipse and batch packager? which is the best way, using RAD or Eclipse with batch Packager in production use? what are pros and cons in both approaches?
    3.which is the best way,BDS generic batch step(implementing BatchRecordProcessor) or custom batch step(implementing batchstepInterface) for using single input and single output streams?

    Thanks in advance
    Please refer to ant scripts in WCG V8 Sample Eclipse Workspace for the latest batch packager examples.

    Now, regarding your other questions:

    1.I know the creation of Ear using batch packager but how to create xjcl using job properties file using batch packager. We did not update the batchsimulator (xJCL generator) capability yet in the V8 workspace. Hopefully, soon.
    2.can we develop any kind and any size of batch application using pojo model using eclipse and batch packager? which is the best way, using RAD or Eclipse with batch Packager in production use? what are pros and cons in both approaches? You can build any kind or size of WebSphere batch app with the Eclipse tools. The benefit of the RAD tooling is a formal batch project type, wizards, and test integration.
    3.which is the best way,BDS generic batch step(implementing BatchRecordProcessor) or custom batch step(implementing batchstepInterface) for using single input and single output streams? BatchRecordProcessor is fine for single in, single out unless you have unique requirements, in which case you are advised to implement BatchJobStepInterface and/or BatchDataStream interfaces directly. There is an article that shows how to do the latter.