Search index process flow

Index build process

Whenever a transactional data is created or modified in Sterling Order Management System Software, the Sterling Search Index senses the need for indexing and sends a subset to outline the transactional data for indexing after the changes are committed in Sterling Order Management System Software.

Starting Sterling Order Management System Software version 9.2.1, the capability to index transactional data is provided for order and shipment entities. If Sterling Search Index is enabled, as part of the business transaction flow, whenever you create or modify an order or shipment, Sterling Order Management System Software checks whether any of the indexable attributes are modified during the transaction and holds the information in memory, and sends an outline with the indexable attributes when the corresponding business APIs or agents are committed.

The Sterling Search Index supports the PeriodicBatchIndexer service, which accumulates the data received from different business transactions and periodically performs a batch update on the index with the information collected up to that point in time. The periodicity time of batch updates is configurable through the yfs.ssi.batch.index.periodicity.secs property in the yfs.properties_ysc_ext property file, which is by default set to 3 seconds. Thus, any changes to the outline data are updated on the index asynchronously, but near real time.

Sterling Order Management System Software is heterogeneous in nature and consists of application server, agent server, and Search Index Server. These servers are deployed outside of Sterling Order Management System Software. As indexing is done asynchronously, there is always a possibility that the requests to the Search Index Server might not succeed, time out, lose track, and so on. Furthermore, by default an index update is triggered only once in 3 seconds. If a server is forcefully stopped, a set of such changes is not submitted to the Search Index Server. Therefore, for each update to the search index a record is inserted in the YFS_Awaiting_Index table with the corresponding EntityName and EntityKey. For example, during an order update a record is inserted in the YFS_Awaiting_Index table with EntityName=Order_Header and EntityKey=<value of OrderHeaderKey>. This record is present in the same database shard as the corresponding entity record.

Sterling Order Management System Software tracks every index update operation in the YFS_Awaiting_Index table. For each indexing attempt that is successful, after the index update operation, the corresponding record is removed from the YFS_Awaiting_Index table. If the index update does not succeed for some reason, for example if the Search Index Server is down or the communication to the Search Index Server does not succeed or times out, an exception is logged and the corresponding records in YFS_Awaiting_Index remains that are later processed by the SSI_Delayed_Sync agent.

The following diagram illustrates the asynchronous index build logic:

The same process is followed when an order is restored from the Order History table.

When transactional data is purged

When an entity data is purged from the main table in Live mode, the corresponding outline document in the Search Index Server is updated as purged.

This activity is triggered from the purge, but processed in a delayed fashion. It inserts a record in the YFS_Awaiting_Index table with EntityName, EntityKey, and ToDelete=Y, which is later processed by the SSI_Delayed_Sync agent. By default, the outline contains the isHistory attribute to determine whether the data is present in transaction table or history table. The outline document derived from a history entity always contains isHistory=Y based on which the outline document in the Search Index Server is updated. It is recommended that you include the required attributes in the outline document as needed for your business to eliminate the dependency on Sterling Order Management System Software. If you want to retrieve detailed information, use the isHistory attribute value to create an appropriate API input to query Sterling Order Management System Software and retrieve detailed information.

When history data is restored

A similar set of processes as followed in the creation of order for building an index are followed when an order is restored from the Order History table. When restore happens the order and related data are rolled back to the transaction tables, and an index update operation is triggered. At this time, the order outline document is derived from the transaction entity, and thus it contains isHistory=N. The new document replaces the previous document that is updated when the order got purged. With this approach, you can track the history and non-history orders.