Data extract overview

The IBM® Sterling Order Management System includes a data extract feature, which you can use to retrieve data from your production environment backup database. For instance, you can retrieve transaction data to include within a data warehouse for further analysis.
The data extract feature is composed of two main components:
  • The data extract configuration service, which is called CocDataExtractConfig.
  • The data extract agent, which is named as CoC Data Extract.

IBM Cloud data extract overview

The data extract capability allows you to export data to an external SFTP server as well as IBM Cloud Object Storage, AWS S3, and other Amazon S3 library compatible external file storage services.

The data extract service reads from a configuration of database tables to extract data and runs corresponding queries against the database. The data extract service creates .csv files and sends them to the file storage service that is owned and managed by the customer. The data extract service runs automatically when triggered by the data extract agent on scheduled intervals. You can also directly run the data extract service, if needed. For example, when you want to re-create a previously output extract file.

The data extract service is run on the production backup database. Therefore, you cannot extract data when the database is under maintenance or when the primary database is not available. Also, you cannot extract data for columns that contain the CLOB data.
Note: As you cannot use the Data Extract feature to extract the BLOB and CLOB data, first create custom tables and populate CLOB data into the non-CLOB columns by using custom agents. Then, use the Data Extract feature to extract data from the custom tables. Ensure that performance testing is done on the custom agents and that the data in the custom table is purged.

In situations, when the data extract service is down, or if the data extract process stops abruptly due to some issues, any missing data is successfully extracted when you again run the data extract service.

The database maintenance or backup time occurs everyday at a configured time early in the morning. During the database maintenance or backup time, the HADR or secondary database is not available. Therefore, ensure that you do not schedule the CoC data extract job during the database maintenance. For more information about the limitations of HADR, see the Db2 documentation.

When the data extract service for your IBM Sterling Order Management System is being configured, ensure that the data extract service runs on its own agent server. By running the service on its own server, the memory calculations that are completed to run the extract jobs are more accurate.

It is recommended that you extract data from the transaction tables. To know the database tables that are supported for data extract and the data extract configuration service, see Data extract configuration service.

Running the data extraction agent in your developer toolkit environment

You can run the data extraction agent in your developer toolkit environment to retrieve data from your backup database.

For more information, see Running the data extraction agent in your developer toolkit environment.