Index search and index build yfs.properties

Use the customer_overrides.properties file to override properties in the Index Search and Index Build category of yfs.properties.

The following table contains the index search and index build yfs.properties and descriptions.

Property Values Description
Index Search and Index Build
api.colony.indexer Valid values = ssi, disabled, nonindexed, <custom value>

Default value = disabled

Defines the logic used to determine the shard from which data needs to be obtained for get-list operations.
  • disabled - Disables obtaining data from other shards.
  • ssi - Uses the out-of-the-box index, Sterling Search Index, to determine the shards from which to obtain data.
  • nonindexed - To be used when sharding is used, but search index is not to be used to determine the shards from which to obtain data. This ensures that data is obtained from all shards.
  • <custom value> - Any custom implementation used to determine the shards from which to obtain data. Use the property api.colony.indexer.custom.class to provide the name of the implementing class.

For example:

api.colony.indexer=ssi

api.colony.indexer.<custom value>.class Valid value = <fully-qualified class name> If api.colony.indexer=<custom value> is defined, provide a fully-qualified class name of the class implementing com.ibm.sterling.afc.colony.indexer.PLTIndexingService.

For example:

api.colony.indexer.<custom value>.class=<fully-qualified class name>

api.colony.indexlookup.processing Valid values = INTERSECTION, <blank>

Default value = INTERSECTION

Some APIs are enabled to search against multiple indices. This property determines the behavior when the input to the API contains attributes from both the indices. If the value is unset, the behavior is to process the index corresponding to the indexLookup element appearing first in the input XML definition. If the value is set to INTERSECTION, a search will be performed against all the indices enabled for the API, and the intersection of the results will be used to determine the colonies from which to obtain data.

For example, the getShipmentList API takes both Order attributes and Shipment attributes as input. Separate indices are maintained for Order and Shipment. Therefore, if getShipmentList is called with both Order attributes and Shipment attributes, setting this property to INTERSECTION ensures that the search is performed on both the indices with the corresponding attributes, and the intersection of colonies returned from each attempt are returned.

For example:

api.colony.indexlookup.processing=INTERSECTION

Note: If api.colony.indexer is set to either disabled or nonindexed, this property is ignored.
Search index properties
yfs.ssi.enabled Valid values = Y, N

Default value = N

Set this property to Y to enable the Sterling Search Index feature. This will index all entities that are enabled for indexing (for example, Order, Shipment).

For example:

yfs.ssi.enabled=Y

Note: This property must be enabled if api.colony.indexer=ssi.
yfs.ssi.search.engine.name Valid values = elasticsearch, <custom value>

Default value = elasticsearch

Determines the name of the search engine used for indexing.

For example:

yfs.ssi.search.engine.name=elasticsearch

Based on the name provided in this property, other properties will be looked up. For example, if the class name property is yfs.ssi.search.engine.name=xyz, provide yfs.ssi.search.engine.xyz.class=<fully-qualified class>.

yfs.ssi.search.engine.<custom value>.class Valid value = <fully-qualified class name> If using a custom implementation (that is, if property yfs.ssi.search.engine.name=<custom value>), provide a fully-qualified class name.

This property defines the class name that enables the custom implementation of the search engine client, where <custom value> is the value defined in the property yfs.ssi.search.engine.name. For example, if yfs.ssi.search.engine.name is defined as xyz, then yfs.ssi.search.engine.xyz.class=<fully-qualified class name>.

yfs.ssi.batch.index.periodicity.secs Valid value = <number of seconds>

Default value = 3

Determines the periodicity with which the search index will be updated through a batch operation. The value provided is the time in seconds after which another batch operation will be submitted to the search engine.

For example:

yfs.ssi.batch.index.periodicity.secs=3

Fail-fast determination logic properties
yfs.ssi.fail.fast Valid values = true, false

Default = true

When set to true, this property enables the fail-fast logic. The fail-fast logic ensures that upon continuous failure, indexing into the search engine and searching from the search engine is temporarily stopped until the problem is resolved. The definition of what constitutes a continuous failure can be configured using the properties that follow.

Continuous failures for indexing and searching operations are tracked separately from each other. In case of continuous failure in either an indexing or search operation, the corresponding operation is not attempted again until the problem is fixed and the operation is re-enabled through an API.

Note: If the indexing operation continuously fails, the search operation is also automatically disabled to prevent searching on an index that is probably dirty.

The successes and failures of both indexing and searching operations are sampled separately on each app-server/agent-server JVM that performs the operation. The sampled success/failure events are used to determine if there has been a continuous failure. The indexing/searching operation is considered to continuously fail under either of the following conditions:

  1. If at least 50 (configurable) search/index operations were performed in a given JVM in the last 10 minutes (configurable), and if there are at least 90% (configurable) of failures in that sample.
  2. If less than 50 search/index operations were performed in a given JVM in the last 10 minutes, and among the last 50 samples, there is a 90% failure (irrespective of how long it took for 50 search/index operations to occur).

For example, if there is a load of 30 index operations per minute overall, and there are five application servers in the cluster to handle that load, each application server would be handling 60 such operations in 10 minutes. In such a case, condition 1 above will apply, and continuous failure will be determined based on 90% failure in this sample of 60 operations, that is, 54 failures out of 60. If, on the other hand, there were 10 application servers handling the same load, there would have been only 30 such operations in 10 minutes. In that case, condition 2 above is applied, and failure is determined only if there are 45 failures (90% of 50) in the last 50 index operations.

When a continuous failure is determined by any one JVM (application servers or agent/integration servers), the same is communicated to all the others and the search/index operations are stopped on all the JVMs. The user is also notified of the same, enabling the user to take corrective measure and re-enable index/search operations.

The three configurable settings described above can be configured using the following properties:

  • yfs.ssi.fail.fast.(index|search).monitor.time.mins
  • yfs.ssi.fail.fast.(index|search).min.sample.count
  • yfs.ssi.fail.fast.(index|search).threshold.percent
yfs.ssi.fail.fast.index.monitor.time.mins

Valid values = <number of minutes>

Default = 10

Determines the minimum duration of time (in minutes) the index operation should fail to be treated as a continuous failure. In the example provided for the yfs.ssi.fail.fast property, this is 10.

For example:

yfs.ssi.fail.fast.index.monitor.time.mins=10

yfs.ssi.fail.fast.index.min.sample.count

Valid values = <number of sample counts>

Default = 50

Determines the minimum sample count to be used to determine a continuous failure situation. In the example provided for the yfs.ssi.fail.fast property, this is 50.

For example:

yfs.ssi.fail.fast.index.min.sample.count=50

yfs.ssi.fail.fast.index.threshold.percent

Valid values = <percentage>

Default =90

Determines the failure threshold that, if crossed, indicates continuous failure. In the example provided for the yfs.ssi.fail.fast property, this is 90%.

For example:

yfs.ssi.fail.fast.index.threshold.percent=90

yfs.ssi.fail.fast.search.monitor.time.mins

Valid values = <number of minutes>

Default = 10

Determines the minimum duration of time (in minutes) the search operation should fail to be treated as a continuous failure. In the example provided for the yfs.ssi.fail.fast property, this is 10.

For example:

yfs.ssi.fail.fast.search.monitor.time.mins=10

yfs.ssi.fail.fast.search.min.sample.count

Valid values = <number of sample counts>

Default = 50

Determines the minimum sample count to be used to determine a continuous failure situation. In the example provided for the yfs.ssi.fail.fast property, this is 50.

For example:

yfs.ssi.fail.fast.search.min.sample.count=50

yfs.ssi.fail.fast.search.threshold.percent

Valid values = <percentage>

Default =90

Determines the failure threshold that, if crossed, indicates continuous failure. In the example provided for the yfs.ssi.fail.fast property, this is 90%.

For example:

yfs.ssi.fail.fast.search.threshold.percent=90

Overrides for specific indexes:

yfs.ssi.fail.fast.(index|search).monitor.time.mins.<index name>

yfs.ssi.fail.fast.(index|search).sample.count.<index name>

yfs.ssi.fail.fast.(index|search).threshold.percent.<index name>

  Set these properties to override for a particular index.

<index name> is the name of the index, such as Order or Shipment. For example, to override the yfs.ssi.fail.fast.index.monitor.time.mins property for the index Order, the property can be set as yfs.ssi.fail.fast.index.monitor.time.mins.Order=10.