Transaction log full error for imports implemented with the Java API

Processing large data sets through imports that are implemented with Java™ API may perform poor and eventually cause Transaction log full error (for example, SQL0964C on DB2®) when data is not committed intermittently.

Symptoms

During the early phase of large imports, entry.save() operations perform fast, but over time decreases significantly. Eventually save operations might fail due to SQL0964C errors (Db2) and scheduler service exception.log files show entries like:

[<code>]2013-01-24 12:08:32,809 [sch_worker_0] ERROR com.ibm.ccd.common.error.AustinException JOB_ID:5613- CWPCM0002E:Generic error / Exception: CWPCO0015E:Update failed, the operation has been rolled back: , Exception:DB2 SQL Error: SQLCODE=-964, SQLSTATE=57011, SQLERRMC=null, DRIVER=3.64.82

com.ibm.db2.jcc.am.SqlException: DB2 SQL Error: SQLCODE=-964, SQLSTATE=57011, SQLERRMC=null, DRIVER=3.64.82

        at com.ibm.db2.jcc.am.bd.a(bd.java:682)

...

        at com.ibm.db2.jcc.am.jo.executeUpdate(jo.java:750)

...

        at com.ibm.ccd.api.catalog.item.ItemImpl.save(ItemImpl.java:698)

...

        at com.ibm.ccd.api.extensionpoints.ImportFunctionArgumentsImpl.run(ImportFunctionArgumentsImpl.java:79)[</code>]

Causes

If no additional transaction handling is implemented, all items that are processed by one import will be handled in one transaction, which in consequence can cause poor import performance and running out of transaction log space on database server side. This is likely to occur only while processing large data sets.

Diagnosing the problem

Add logging statements to your import extension point code to monitor progress in data processing and check exception.log files for errors.

Resolving the problem

Implement transaction handling in your import extension point code and commit data at regular intervals. For more information, see:

Sample code for transaction and exception handling in import and report jobs
Transactions and exception handling guidelines for Java APIs: Common import scenarios