Index synchronization

When running correctly, indexes are synchronized. However, there are several scenarios where an index can be out of sync.

An index is said to be in a consistent state when it is a true representation of its corresponding entity’s database state. For example, the ‘Order’ index is consistent if it is up to date with the contents of the YFS_ORDER_HEADER and related tables. In this case, the index is a true representation of the database state, since a query on the index returns the same set of records as the database tables. This is critical for proper functioning of the search APIs in Order Search. If an order is updated through an API, but the corresponding index is not updated, the order may not be returned as part of search API calls because the search API relies on the index to fetch the orders.

Since index update operations are done asynchronously after a small delay, and because there is a possibility that the operation might fail when attempted, the search index may not be strictly consistent all the time. However, if you enable Sterling Search Index and is active, every index-update operation is attempted in near real-time. Failures are notified and tracked through the YFS_Awaiting_Index table and later attempted again by the SSI_DELAYED_SYNC agent. Therefore, though not strictly consistent all the time, if Sterling Search Index is enabled, the index can be assumed to be ‘in sync’ with the database for all practical purposes under steady use.

However, there are scenarios where index might get considerably out of sync. The following are some such scenarios:

  • After an upgrade, there are several orders already in the system that are not indexed. As these orders are created when Sterling Search Index was not active, they are not tracked anywhere, not even in the YFS_Awaiting_Index table. Therefore, the index is out-of-sync.
  • Pre-existing orders: The Sterling Search Index feature is enabled after several orders are created. For example, in case of a fresh installation when the feature is not enabled upfront.
  • The outline template is modified to add more indexable attributes. If the new attribute corresponds to a column that is already present in the database, and have values populated for pre-existing records, the index is now out of sync. For example, if the ADDRESS_ID field is added to the outline template and if there are existing orders with the value populated in this field, then searching for orders with ADDRESS_ID in the search criteria might return only partial records.
  • The Sterling Search Index feature is disabled for some time. In such a case, Sterling Search Index does not know which orders are created when the feature had been disabled.
  • Orders are migrated to Sterling Order Management System Software without using the Sterling Order Management System Software APIs. As Sterling Search Index uses Sterling Order Management System Software APIs, Sterling Search Index will not know of such orders. The index becomes out of sync.

To track ‘in sync’ orders, use the YFS_Index_Sync statistics table. Since the order entity is enterprise-specific, this table holds the ‘InSync’ flag for each enterprise and each index. When an index is out of sync for any of the mentioned reasons, searching from the index becomes unreliable. You can manage the enterprise-specific synchronization status through the Index Management Console.

In some scenarios, the 'InSync' flag is automatically set to ‘N’ by Sterling Search Index. For such scenarios, take the following actions:

  • When an application, agent, or integration server is started with the Sterling Search Index feature disabled (yfs.ssi.enabled=N), all enterprises are marked out of sync for all indexes. This is because, as the Sterling Search Index feature is disabled it is not possible to track orders that are created or modified and need to be indexed. After you enable this feature, run the SSI_MASS_SYNC mode in ‘unindexed’ mode for each enterprise for order index, and then mark all enterprises to be in-sync through the Index Management Console.
  • When the outline template definition is changed for order entity, all enterprises are marked not-in-sync for the order index. Evaluate the impact of the outline template definition update to check which enterprises are impacted through this change. For each enterprise that is impacted, run the SSI_MASS_SYNC agent in ‘versionsync’ mode for that enterprise and index to reindex all orders of that enterprise with the new outline template definition. For each enterprise that is not impacted or for which the agent is run, mark the enterprise to be in-sync by using the Index Management Console.
  • When an application, agent, integration server is started for the first time after an upgrade, all enterprises are marked out of sync for all indexes. This is because, there can potentially be pre-existing orders that need to be indexed. Run the SSI_MASS_SYNC agent in the ‘unindexed’ mode for all enterprises (for order index) that have pre-existing order records. Then, use the Index Management Console to mark each enterprise to be in sync for order index.

If you have to manually mark an enterprise as not in sync, you can use ‘Mark As Not Synchronized’ in the Index Management Console. You may need to do that under exceptional circumstances, such as if an index gets corrupted and the entire index needs to be rebuilt. Basically, until the index is rebuilt in these exceptional cases a search attempt from Order Search is either unreliable or may result in errors. So, users can depend on the ‘InSync’ flag as an indicator to decide whether a search in Order Search is reliable or retrieve the order data directly from Sterling Order Management System Software. A custom implementation to establish this behavior from an e-commerce system is a good solution to make the search result always reliable.

Note:
  • The index operations continue irrespective of the 'InSync' flag status. This becomes an indicator for client applications to rely on the data in Order Search.
  • If the 'InSync' is automatically set to ‘N’ by Sterling Search Index, you are notified through an alert with either an SSI_WARN_01 or SSI_WARN_02 error code.