Incremental data load

Incremental data load refers to changes in the OLTP source system. Every change in the OLTP source system must be reflected in the data mart system, and occurs through incremental load (basically, synchronizing the data between the OLTP source system and the data mart system). You must also add the configuration data and master data because they do not change frequently.

Loading incremental data into the staging database

If you have deployed a sharded environment, it is recommended that you create a staging database. The staging database contains tables that extract data from the configuration, master, and transaction tables. For incremental data load, the tables that extract the configuration data and the master data are the same as that of the initial data load. The tables that extract transaction data are:
  • YFS_ORDER_HEADER
  • YFS_ORDER_LINE
  • YFS_ORDER_RELEASE_STATUS
  • YFS_ORDER_DATE
  • YFS_ORDER_LINE_SCHEDULE
  • YFS_CHARGE_TRANSACTION
  • YFS_PAYMENT
  • YFS_ORDER_HOLD_TYPE
  • YFS_ORDER_HOLD_TYPE_LOG
  • YFS_ORDER_AUDIT
  • YFS_ORDER_AUDIT_LEVEL
  • YFS_SHIPMENT
  • YFS_SHIPMENT_LINE
  • YFS_OPPORTUNITY
  • YFS_ORDER_TS_TAG
  • YFS_SHIPMENT_TS_TAG
  • YFS_PERSON_INFO
  • YFS_LINE_CHARGES
  • YFS_OPPORTUNITY_TS_TAG

If you are using InfoSphere® DataStage®, refer to IBM InfoSphere Information Server documentation for more information about loading data into the database. If you are using any other ETL tool, refer to the corresponding product documentation for more information.