Enabling and configuring the Federated Data Repository process indexing

Edit online
Draft comment:
This topic only applies to BAW, and is located in the BAW repository. Last updated on 2025-03-13 12:15

Business Automation Workflow can index data about process instances and tasks in the federated data repository.

To enable the Federated Data Repository process indexing, meet the prerequisites and then complete the following configuration on your Business Automation Workflow servers.

Before you begin

Complete configuring the connection to the Federated Data Repository, as described in Configuring the Federated Data Repository connection.

Important: Configure the Federated Data Repository to prevent the automatic creation of the index that stores the documents about process instances and tasks. In Elasticsearch and OpenSearch, the configuration can be done through the action.auto_create_index setting. See the specific documentation for Elasticsearch External link opens a new window or tab and OpenSearch External link opens a new window or tab.

Configuring Business Automation Workflow to index into the Federated Data Repository

Edit online

After you configure Business Automation Workflow to communicate with the Federated Data Repository, you can enable and configure the Federated Data Repository process indexing by editing the 100Custom.xml configuration file.

About this task

You can enable the Federated Data Repository process indexer by setting the <fdr-index-enabled> property to true as shown in the following example:
<properties>   
    <server>
      <search-index merge="mergeChildren">
            <fdr-index-enabled merge="replace">true</fdr-index-enabled>
      </search-index>
    </server>
</properties>

To tune process indexing, you can define other optional properties. See the properties that you can use to customize the Federated Data Repository process indexing behavior in the following table.

Table 1. Properties that customize the behavior of the Federated Data Repository process indexing
Property Description Default value
fdr-index-enabled

A flag that indicates whether Federated Data Repository process indexing is enabled.

For improved performance, easier maintenance, and richer search capabilities, enable Federated Data Repository process indexing by setting <fdr-index-enabled> to true, <task-index-enabled> to false, and <fdr-use-instead-of-lucene> to true. This ensures features use Federated Data Repository data instead of the Lucene-based Process Portal index.

 false
fdr-use-instead-of-lucene

When Federated Data Repository process indexing is enabled with <fdr-index-enabled> set to true, it is recommended to also set <fdr-use-instead-of-lucene> to true so that Business Automation Workflow searches the Federated Data Repository instead of the Lucene-based Process Portal index.

Considerations when setting <fdr-use-instead-of-lucene> to true:

  • Searches against the Federated Data Repository can be affected by limitations that are configured in the Elasticsearch or OpenSearch instance that is used as the Federated Data Repository.
    For example, the index.max_result_window property defines the maximum from + size value for index searches, where from is the starting offset and size is the number of results to return. The default value is 10000.
  • The /rest/bpm/wle/v1/tasks REST API search definition includes the list of fields to return. Each field has an input field name (used in the search definition), and an output field name (used in the response for the list of matching tasks). In the search response, the attributeInfo JSON array contains the mapping between the input and output name for each field.
    You can also provide an additional filter by using the queryFilter query parameter:
    • With the Lucene-based Process Portal index, fields in the query filter can be referenced by using either the input or output field name.
    • With the Federated Data Repository, only the input field name can be used.
  • With the /rest/bpm/wle/v1/tasks REST API, you can provide an additional filter by using the searchFilter query parameter. When searching the Federated Data Repository, the following variations of the search filter syntax are not supported:
    • using ~ in the middle of the search filter.
    • escaping the wildcard characters ? and {} with {*}.
  • With the /rest/bpm/wle/v1/tasks REST API, you can specify the task status to search for by using the interaction query parameter:
    • The Lucene-based Process Portal index accepts the following values: WORK_ON, WORK_ON_ACTIVE, ASSESS_AVAILABLE, ASSESS_AND_WORK_ON, CHECK_COMPLETED, and BROWSE_ALL.
    • With the Federated Data Repository, WORK_ON_ACTIVE is not supported.
  • Searches that rely on the Federated Data Repository index never return the CONTAINMENT_CTX_ID field.

Once you have set <fdr-use-instead-of-lucene> to true and verified that Process Portal dashboards and the /rest/bpm/wle/v1/tasks REST API work as expected, disable the Process Portal index by setting <task-index-enabled> to false to prevent unnecessary database access to maintain the Lucene-based Process Portal index, which is no longer needed.

 false
fdr-index-name
Enables you to define the name of the Federated Data Repository index that stores the documents about process instances and tasks. The index name is prefixed with “@”, and the index alias is named exactly as the <fdr-index-name> value.
Important: To prevent the automatic creation of the index in Elasticsearch and OpenSearch, configure the Federated Data Repository by using the action.auto_create_index property.

Do not define the fdr-index-name property if you want to federate the Business Automation Workflow system from Process Federation Server that runs as part of Cloud Pak for Business Automation or as part of a stand-alone Business Automation Workflow deployment on containers.

 If this property is not defined, the index name is created by using theBusiness Automation Workflow system unique identifier. For example, “@068e7d66-b043-490e-8ee8-9e92be6c4ce6”.
fdr-index-shards The number of shards to use when you create the Federated Data Repository index. 3
fdr-index-replicas The number of replica shards to define for the Federated Data Repository index. This property is only used by Business Automation Workflow when it creates the Federated Data Repository index. To change the number of replica shards on the existing Federated Data Repository index, use the Elasticsearch or OpenSearch API directly. 2
fdr-index-threads The number of indexing threads per node. 1
fdr-index-interval The minimum interval between two starts of the indexing cycle, in milliseconds. 1000
fdr-index-house-keeping-interval The interval between two housekeeping maintenance operations, in minutes. If this value is negative, the housekeeping maintenance operation is not triggered internally, and you must manually run the operation regularly instead. 60
fdr-index-house-keeping-audit-max-age The maximum age (in minutes) of indexing audit entries to keep (to not delete) on a housekeeping maintenance operation. 10080 (equals to 7 days)
fdr-index-house-keeping-record-max-age The maximum number of maintenance operation records of each type to keep (to not delete) on a housekeeping maintenance operation. 50
fdr-index-batch-size The maximum size of indexing queue entries to process on a single indexing cycle. 1000
fdr-index-deletion-batch-size The maximum number of rows to delete when a maintenance operation requires row deletions. For example, during a housekeeping maintenance operation. 50000
fdr-index-reconciliation-batch-size The maximum number of discrepancies to identify on each cycle of the “reconcile” maintenance operation. 500
fdr-index-failure-threshold Specifies how many times a failed operation such as the creation, update, or deletion of a document in the Federated Data Repository index can be tried again. When the number of retries reaches the threshold, the operation request remains in the queue but is ignored by the indexer. To attempt processing the queue entry again, you must run the "retry-failures" maintenance operation. 20
Back to top

Migrating from Process Federation Server indexing to Federated Data Repository indexing

Edit online
If you are using Process Federation Server to index process data with the <ibmPfs_bpdIndexer> configuration, you must migrate to Federated Data Repository indexing for Business Automation Workflow 24.0.0.0 and later versions.

Before you begin

Before migrating from Process Federation Server indexing to Federated Data Repository indexing:

About this task

The migration process involves enabling Federated Data Repository indexing on your Business Automation Workflow system and removing the deprecated <ibmPfs_bpdIndexer> configuration from Process Federation Server. This ensures that process indexing is performed directly by Business Automation Workflow rather than through Process Federation Server.

Important: Plan for a maintenance window during the migration. While the migration can be performed with minimal downtime, search functionality may be temporarily unavailable during the transition.

Procedure

  1. From the Business Automation Workflow system, enable Federated Data Repository process indexing.
    Follow the configuration steps in Configuring Business Automation Workflow to index into the Federated Data Repository to enable Federated Data Repository indexing on your Business Automation Workflow servers.
  2. Verify that Federated Data Repository indexing is working correctly.
    1. Monitor the Business Automation Workflow logs for successful indexing operations.
    2. Verify that new process instances and tasks are being indexed to the Federated Data Repository.
    3. Test search functionality to ensure that newly indexed data is searchable.
  3. From the Process Federation Server system, stop all Process Federation Server servers.
  4. Remove the <ibmPfs_bpdIndexer> configuration from the Process Federation Server server.xml file.

    Locate and remove or comment out the <ibmPfs_bpdIndexer> element and its associated configuration. For example:

    <!-- Removed: PFS indexing deprecated as of BAW 24.0.0.0
<ibmPfs_bpdIndexer
   federatedSystemRef="bpm1"
   dataSourceRef="bpd_db2"
   indexingInterval="1000"
   ... />
-->

    For information about the <ibmPfs_bpdIndexer> element and its properties, see Configuration properties for the Process Federation Server index.

  5. Remove the change log tables from the Business Automation Workflow database.

    The change log tables that are created for Process Federation Server indexing are no longer needed when using Federated Data Repository indexing. You can optionally remove these tables to free up database space:

    • LSW_PFS_BPD_CHANGE_LOG
    • LSW_PFS_BPD_CHANGE_LOG_CONSUMER
    Note: Before removing these tables, ensure that Federated Data Repository indexing is fully operational and that you have verified that search functionality is working correctly.
  6. Update the Business Automation Workflow configuration to disable the Lucene-based index.

    After confirming that Federated Data Repository indexing is working correctly, disable the Lucene-based Process Portal index by setting <task-index-enabled> to false in the 100Custom.xml file. This prevents unnecessary database access to maintain the Lucene-based index.

  7. Restart the Process Federation Server servers.

Results

After completing the migration, your Business Automation Workflow system indexes process data directly to the Federated Data Repository, and Process Federation Server no longer performs indexing operations. This configuration provides improved performance and easier maintenance.

What to do next

After migration:

Back to top