Data synchronization guidelines

In a deployment model where remote systems communicate with a corporate server network, outages may occur. Sterling™ Order Management System Software provides a data synchronization feature that allows you to synchronize configuration data and master data from one environment to another. Data synchronization involves these major tasks:

Export data from the source database
Import data into the target database

This section provides data obtained from the following data synchronization operations:

FULL Export
DELTA Export
Import data from the FULL Export
Import data from the DELTA Export

This data is provided as an example that you can use as a general guideline when planning your own data synchronization export and import operations. Actual results may vary. The section also provides recommendations about performing an export and an import.

Setup data for FULL synchronization

The following setup data was used for the FULL synchronization.

3000 stores
1.4 million items in the catalog
70,000 users
160,000 pricing rules
40,000 pricelist lines for one store

Setup data for DELTA synchronization

The following setup data was used for the DELTA synchronization.

150 stores (5% of the 3000 stores)
70,000 items (5% of the 1.4 million catalog)
3500 users (5% of the 70,000 users)
1600 pricing rules (10% of the 160,000 users)
4000 pricelist lines for one store (10% of the 40.000 pricelist lines)

FULL export: time estimates and hard drive space

Table 1 provides the time estimates and hard drive space for a FULL export, based on the data in Setup data for FULL synchronization.

Table 1. FULL export
Profile	Time	Hard Disk Size (uncompressed/compressed)
Configuration	30 min.	90MB/10MB
Catalog	1 hour 30 min.	16GB/500MB
User	10 min.	125MB/10MB
Pricing rule	10 min.	520MB/25MB
Pricelist line (one store)	10 min.	25MB/1MB

Import data from the FULL export: time estimates

Table 2 provides the time estimates for the import of data from the FULL export described in Table 1.

Table 2. Import data from the FULL export
Profile	Time
Configuration	40 min.
Catalog	1 hour
User	10 min.
Pricing rule	30 min.
Pricelist line (one store)	10 min.

DELTA export: time estimates and hard drive space

Table 3 provides the time estimates and hard drive space for the DELTA export, based on the data in Setup data for DELTA synchronization.

Table 3. DELTA export
Profile	Time	Hard Disk Size (uncompressed/compressed)
Configuration	5 min.	1MB/30KB
Catalog	10 min.	65MB/2MB
User	5 min.	1MB/30KB
Pricing rule	5 min.	2MB/75/KB
Pricelist line (one store)	5 min.	2 MB/100KB

Import from the DELTA export: time estimates

Table 4 provides the time estimates for the import of data from the DELTA export described in Table 3.

Table 4. Import data from the DELTA export
Profile	Time
Configuration	5 min.
Catalog	10 min.
User	5 min.
Pricing rule	5 min.
Pricelist line (one store)	5 min.

Recommendations for full export

This section provides recommendation for running a FULL export.

Depending on the size of the data to export, the FULL export may require at least a 4GB JVM.
For a very large data set, consider increasing the number of messages and decreasing the locking entities per file on the agent criteria. However, do not set the locking entities per file to 1. This configuration ensures that less data is stored in each XML file and alleviates any database locks that might occur.
Depending on your data set, the FULL export might require a large amount of disk space on the machine where the agent is running. After the export is complete, the XML files are compressed to about 5-to-10 percent of their original size.
During the first part of the FULL export, the database can reach a CPU utilization of 50 percent or higher, and depending on your data, up to a 30 percent disk IO utilization.
Full exports should be run on quiet systems.

Recommendations for full import

This section provides recommendations for running a FULL import.

Depending on your data set, imports might require a large amount of disk space. The import first uncompresses all exported files, reclaiming the original disk space.
To improve database performance, disable entity change tracking on the target system.
Depending on the size of your data and the size of your generated XML files, you might need to increase the space of your database transaction/undo log because commits are executed at the end of every XML file.