Common import scenarios
The following scenarios cover some highly-used use cases for import jobs.
Data migration and initial load imports
The data migration and initial load imports tend to be very large that consist of item data that come from any number of sources. Most often, a single flat file containing all items and attributes is loaded into IBM® Product Master. The initial load files are typically processed to optimize the data format for faster imports, like sorting the file so that multiple references to a single item are sequential in the file. Since these are one time loads, human intervention is sometimes required to perform the optimizations. You can use CSV because it has reduced file size and overhead compared to formats such as XML. The initial load input file spec should be based on the data model. To reduce implementation work, this format could also be used for export of all Product Master data that provides a single interface for import and export. These imports are typically started manually.Incremental batch imports (multi-item)
The incremental batch imports contain less data than an initial load but follow the same principal of a single file to update or add many items in batch. These are typically scheduled to run at standard intervals (nightly or weekly) depending on business needs and pick up files produced from other systems on a corresponding interval.Transactional imports (single item)
The transactional imports are single record deltas that are received by Product Master and processed more frequently than the incremental batch imports. These typically contain XML messages and are triggered by:- Web Services
- An update is sent directly to Product Master through Web services and is processed in real-time. Web services are ideal for small request and response transactions. They do not scale to support large file transfers.
- Scheduled poll of message queues
- A scheduled import or report will poll a message queue at a standard interval depending on business needs (15 minutes, 30 minutes, 1 hour). The job will pick up messages from a queue and process them in the order received.