Topic
2 replies Latest Post - ‏2013-01-19T14:47:50Z by LindyHopper
LindyHopper
LindyHopper
17 Posts
ACCEPTED ANSWER

Pinned topic How to write very long running commands

‏2012-12-18T14:03:19Z |
I have a requirement to upload 10s of thousands of lines from a file to an external system via web service calls (roughly 1 call per line of data in the file).
The user uploads the file of data into the system via an entry screen and I have been looking at ways of running a background command to do the web service upload, so the screen doesn't block.
I tried the recommended AddJobCmd approach as per the Info Center documentation, but because the command will run for an hour or so, I get the warnings in the log suggesting that the thread might be hung.
I looked at setting using a work manager (starting with default) and setting...

addJobCmd.setApplicationType(
"default");

...and adding an applicationType tag to wc-server.xml...

<component compClassName=
"com.ibm.commerce.scheduler.SchedulerComm" enable=
"true" name=
"Scheduler"> <property autoClean=
"off" broadcastExpireTime=
"1800" contextSetName=
"Authoring" cycleTime=
"600" display=
"false"/> <applicationType applicationName=
"default" maxNumofThreads=
"10"/> </component>


...before firing off the command, but couldn't work out why it still gave the warnings every 10 minutes or so as the default WorkManager 'Work timeout' value was 0, which I presumed meant infinite.
There doesn't seem to be much documentation on how to write a very long running command, so I'm having to resort to dividing the work up and adding jobs for each section. Not ideal as it makes the logic more complex than just running a single command for the duration.

Does anyone have a good approach to solving this problem?

Regards
Updated on 2013-01-19T14:47:50Z at 2013-01-19T14:47:50Z by LindyHopper
  • Raj.S
    Raj.S
    427 Posts
    ACCEPTED ANSWER

    Re: How to write very long running commands

    ‏2012-12-18T18:33:19Z  in response to LindyHopper
    Hi,
    Can you try the following approaches:
    Approach 1:

    I have not tried this approach myself but I believe this should work.

    Add a new attribute transactionTimeout="0" for the following component. 0 - value will override the maximum timeout to Infinite.
    <component compClassName="com.ibm.commerce.scheduler.SchedulerComm" enable="true" name="Scheduler">
    <property autoClean="off" broadcastExpireTime="1800" transactionTimeout="0"
    + conntextSetName="Authoring" cycleTime="600" display="false"/>+
    </component>
    Follow the below link for more details:
    http://pic.dhe.ibm.com/infocenter/wchelp/v7r0m0/topic/com.ibm.commerce.admin.doc/concepts/cjsconfig.htm
    Approach 2:

    If approach 1 doesnot work, you can do the following.
    By default the webcontainer transaction timeout is 300 seconds and hence every controller command timesout for every 300seconds. You will have to handle the transactions explicitly in the custom command created.

    Have a logic to split your business logic and executes in different transactions. Some pseudo code below:

    public void performExecute() throws ECException {
    + // TODO Auto-generated method stub+
    + super.performExecute();+
    + TransactionHandle txhandle = null;+
    + try {+
    + int counter = 0;+
    + txhandle = com.ibm.commerce.server.TransactionManager.begin();+
    + +
    + // Read File+
    + // Loop running over the file+
    + for (int i=0; i<=numberofLines; i++) {+
    + +
    + // Make webservice call - 1 second+
    + counter++;+
    + +
    + if (counter == 80) {+
    + com.ibm.commerce.server.TransactionManager.commit(txhandle);+
    + txhandle = com.ibm.commerce.server.TransactionManager.begin();+
    + counter = 0;+
    + }+
    + +
    + }+
    + com.ibm.commerce.server.TransactionManager.commit(txhandle);+
    + } catch (Exception e) {+
    + com.ibm.commerce.server.TransactionManager.rollback(txhandle);+
    + e.printStackTrace();+
    + }+
    + +
    + +
    + }+
    Let us know if this helps.

    Rgds,
    Raj
    • LindyHopper
      LindyHopper
      17 Posts
      ACCEPTED ANSWER

      Re: How to write very long running commands

      ‏2013-01-19T14:47:50Z  in response to Raj.S
      Not being able to change the settings in wc-server.xml, we eventually went with something similar to option 2.

      We submit the batch command and process a block of work that we know can be processed within the 10 minutes allowed for a command before it is declared hung.
      Once the first block has been completed successfully, another job is submitted to handle the next block of work and so on until all the work is completed. We pass an ID generated by the launcher from job to job, so the final job can update a status.
      It's clumsy, but is working reliably so far.

      Thanks