External custom full sync support

In some OMS environments, because of the high volume of the data, initial full sync of the catalog and related sync profiles is done manually outside the sync agents using the native database features. Delta-sync for the same sync profiles is done using the data-sync feature of the product.

Currently, without this feature manual changes are done with each deployment as if the full-sync is performed. This is not scalable for a large number of stores.

This use case is addressed in the following section:

Defining Sync Profiles:

When defining the Sync Profiles from the SBC UI or through the APIs, a user can specify Sync Type in the sync profile definition to specify whether the profile will be used for:

  • Full-sync
  • Delta-sync, or
  • Blank (both the full-sync and delta-sync)

When the full-sync or delta-sync is performed with the CHANGE_DATA_EXPORT agent, the agent queries all the profiles to find the entity changes which need to be exported.

If Sync Type is selected as Full or Delta during the Sync Profile definition, the CHANGE_DATA_EXPORT agent will only consider the sync profiles whose SyncType matches the CHANGE_DATA_EXPORT agent Sync Type which is passed in the agent criteria parameters.

So with this, for the entities containing a high volume of records a system user can perform the full-sync externally and set Sync Type for the sync profiles for the data as ‘Delta’ such that these sync profiles are only considered for the delta-sync by the CHANGE_DATA_EXPORT agent.

MarkEntityChangesForFullSync

Once the external full sync is complete, the service MarkEntityChangesForFullSync can be used by a system user to mark the entity changes the source environment as sync exported. So, the entity changes made after the external full sync are not lost due to optimization.

Using this service, a user can pass the identifiers for the entity or the database table or sync profiles in the API input that are exported externally by a system user. This input is analyzed to identify the eligible entity changes to be marked as sync exported. The service input samples are following:

OrganizationCode: the organization associated with the sync profile or the entity change records.

SyncProfileID: The ID for the Sync Profile.

EntityID : The database table name or the locking entity for the entity change records.

Sample Configuration: Sync profiles 'SyncProfile1' and 'SyncProfile2' are configured for the organization code 'DEFAULT'.

Sample 1: Passing the sync profile identifiers in the input

<markEntityChangesForFullSync OrganizationCode="DEFAULT">              
      <SyncProfileList>                              
      <SyncProfile SyncProfileID="SyncProfile1"/>                              
      <SyncProfile SyncProfileID="SyncProfile2"/>              
      </SyncProfileList>
      </markEntityChangesForFullSync>

Sample 2: Passing the identifiers for the locking entities in the input

<markEntityChangesForFullSync OrganizationCode="DEFAULT">              
      <EntityList>                              
      <Entity  EntityID="YFS_ITEM"/>              
      </EntityList>
      </markEntityChangesForFullSync>

Sample 3: Passing the identifiers for the locking entities and sync profiles in the input

<markEntityChangesForFullSync OrganizationCode="DEFAULT">              
      <SyncProfileList>                              
      <SyncProfile SyncProfileID="SyncProfile1"/>                              
      <SyncProfile SyncProfileID="SyncProfile2"/>              
      </SyncProfileList>              
      <EntityList>                              
      <Entity  EntityID="YFS_ITEM"/>              
      </EntityList>
      </markEntityChangesForFullSync>