Querying bulk load status

When you use the Pack to load large amounts of data in bulk to Salesforce.com, it is possible that the data might not be completely processed by Salesforce.com before the IBM® InfoSphere® DataStage® job has finished running. For example, batches of data might be in several states when the job completes: Queued, InProgress, Completed, or even Failed. To ensure that the data has been completely processed by Salesforce.com and is ready to use, you can use the Pack to query the status of the bulk load job.

You can create queries that return the following details:
  • SFDC job ID: When Salesforce.com receives data in bulk, it assigns a Job ID to the entire set of data. For example: 75070000000PCpk. Job IDs can be obtained in one of following ways:
    • From the Job ID File specified in a Pack bulk load job.
    • From the Salesforce.com portal by selecting Setup > Administration Setup > Monitoring > Bulk Data Load Jobs.
  • SFDC batch ID: Data within an SFDC job can be split into multiple batches. Salesforce.com assigns an ID to each batch. For example: 75170000000PIHFAA4.
  • SFDC record ID: Salesforce.com assigns an ID to each record it processes. For example: a0370000006dZjmAAE.
  • SFDC job status: When Salesforce.com processes a job, it assigns one of the following statuses to that job: Open, Closed, Aborted, or Failed.
  • SFDC batch status: When Salesforce.com processes a batch, it assigns one of the following statuses to that batch: Queued, Completed, InProgress, Failed or Not Processed.
  • SFDC record status: When Salesforce.com processes a batch, it assigns one of the following statuses to the records in the batch: Success or Failed.
You can create the following types of status queries:
Job status query
Returns the SFDC job ID and status of a Salesforce.com job. For example, this type of query might return something like the following result:
"75070000000PCau","Closed"
Batch status query
For each batch in an SFDC job, returns the SFDC job ID, the SFDC batch ID, and the SFDC batch status. For example, this type of query might return something like the following result:
"75070000000PCau","75170000000PIHFAA4","InProgress"
Record status query
For each record in an SFDC job, returns the SFDC job ID, the SFDC batch ID, the SFDC record ID, and the SFDC record status. For example, this type of query might return something like the following result:
"75070000000PCaz","75170000000PIHKAA4","a0370000006dZjmAAE","Success"
The following examples of status query jobs are provided in the BulkLoadStatusCheckingExample.dsx project file that is located in the examples directory on the Microsoft Windows installation image for the Pack for Salesforce.com.
Table 1. Example status query jobs in BulkLoadStatusCheckingExample.dsx
Example status query job Description
example1 Demonstrates a basic job status query.
example2 Demonstrates a basic batch status query.
example3 Demonstrates a basic record status query.
example4 Demonstrates a basic bulk load job.
example5 Demonstrates the use of a sequence job for bulk load and status query.

To query bulk load status, you can either use an imported example job, or design a new job.

Tip: In general, you should consider placing your bulk load jobs and associated queries within an InfoSphere DataStage sequence job. This allows you to easily manage the workflow between the bulk load job and the status query. Specifically, it makes it easy for you to have the bulk load job place an SFDC job ID in a specific file, and then provide that file directly to the status query. (Of course, other facilities are available in sequence jobs for error handling, conditional branching, variable substitution, and so on.) For a basic example, see example5.