Connecting to an Elasticsearch content repository

Connect your agent to a content repository on Elasticsearch to enable intelligent retrieval of documents, articles, or knowledge base entries. By integrating with Elasticsearch, your agent can search and return relevant content during user interactions, improving response accuracy and usefulness.

Connect your agent to Elasticsearch

Follow the steps to connect to an Elasticsearch instance:

Note: The Choose knowledge + option does not appear if a different knowledge source is already connected. To switch sources, click Replace source, select your new source, confirm by clicking I understand. When you replace the source, all previous repository details and files are deleted and cannot be recovered.

On the agent configuration page, go to the Knowledge section.
Click Choose knowledge +.
Under Add source, click New knowledge.
Select Elasticsearch and click Next.
Under Elasticsearch url, enter the endpoint URL of your Elasticsearch cluster.
In Elasticsearch port (optional), enter the port number to connect to the Elasticsearch cluster.
From Choose an authentication type, select Basic authentication or API key.
- For Basic authentication, provide a username and password.
- For API key, provide the Elasticsearch API key.
Click Next to proceed. Configure the search index that you want to access from your Elasticsearch instance and map the fields that are used in the search results.
In the Elasticsearch index, enter the Elasticsearch index name. Multiple indexes are not supported.
In the Configure result content section, map the fields of the specified index to the corresponding fields in the search response:
1. Title: Search result title. Use the title, name, or similar type of field from the collection as the search result title.
2. Body: Search result description. Use an abstract, summary, or highlight field from the collection as the search result body.
3. URL: This field can be populated with any footer content that you want to include at the end of the search result.
When you configure the query body in the Advanced Elasticsearch Settings to search the nested documents, you must ensure that the Title, Body, and URL are from the fields of the inner documents in your Elasticsearch index. For more information about using nested queries, see Configuring the query body for Elasticsearch.

For a detailed procedure on Advanced Elasticsearch settings see, Set up custom filters and query body.
Click Next to proceed.
In the Knowledge details section,
- Name: Enter a display name for the knowledge source.
- Description: Provide a short description. For more information, see Providing context about the knowledge to the agent.
Save your settings.

Update Elasticsearch settings

To change the settings of the Elasticsearch content repository:

Go to the Knowledge section.
Click the vertical ellipsis (⋮) next to the selected knowledge source.
Select Edit details and update the configuration.
Save your settings.

Delete Elasticsearch repository

To delete an Elasticsearch content repository:

Go to the Knowledge section.
Click the vertical ellipsis (⋮) next to the selected knowledge source.
Click Remove.

Set up custom filters and query body

Configure a custom filter to query results and a query body for Elasticsearch in the Advanced Elasticsearch settings section. This configuration is optional.

Configure custom result filters

In the Configure the custom result filter for Elasticsearch (optional) field, you define the filter as an array of objects so that you can create filters to arrange the content per the query body. For more information, see Configuring custom filters for Elasticsearch.

Configure query body

In the Configure the query body for Elasticsearch (optional) field, enter the query body.

The query body is used to manipulate the user requests into a format that the search expects. It controls the query forms, search fields, filters, and query size. In the REST API, the query body is an object that represents the POST body for the _search request to Elasticsearch. The query body has a "$QUERY" token to represent the customer's query, and a "$FILTER" token to represent the array of filters that are defined either in the search settings or at the step level.

By default, the integration with Elasticsearch uses keyword search, but you can configure the query body as an advanced Elasticsearch setting in your agents to enable more advanced search techniques, such as:

Semantic search with ELSER.
k-nearest neighbor (kNN) dense vector search.
Nested query to search nested documents.
Hybrid search.
Search on a semantic text field.

For more information about using different types of query body examples, see Configuring query body for Elasticsearch.

For more information about the Elasticsearch _search API request body, see Elasticsearch search API request body Icon for redirecting to external pages. in Elasticsearch documentation.

Pass context variables

You can make your filters dynamic by referencing context variables in your Elasticsearch filter templates. This enables runtime substitution of values based on user or session context, making your queries more flexible and personalized.

How it works

In your content repository, set up filters and reference context variables in the format: {context_var}
When the agent runs, it fetches context from:
- The runs endpoint payload, or
- The JWT token associated with the session
The knowledge runtime replaces the context variables in your filter template with actual values from the context

Examples for Elasticsearch filter substitution

Each example includes:

filter: A template with references to context variables, entered in the filter field for dynamic substitution.
final_filter: The template after replacing context variables with actual values. The query is sent to Elasticsearch for data retrieval.
context_variable: A set of key-value pairs defining the context variables used in substitution.

Example 1:

    filter = [{"bool": {"should": [{"match": {"title": "{location}"}},
                                     {"match": {"text": "{name}"}},
                                     {"match": {"id": "{title}"}}]}}]
    context_variable = {"location": "US", "name": "Pooja", "title": "developer"}
    final_filter = [{"bool": {"should": [{"match": {"title": "US"}},
                                              {"match": {"text": "Pooja"}},
                                              {"match": {"id": "developer"}}]}}]

Example 2:

    filter = [{"terms":{"title":["dialog","{topic}"]}}]
    context_variable = {"topic": "action"}
    final_filter = [{"terms":{"title":["dialog","action"]}}]

Example 3:

    filter = [{"bool": {"should": [{"match": {"title": "{location}"}},
                                     {"match": "{name}"}
                                     ]}}]
    context_variable = {"location": "US", "name": {"text": "Pooja"}}
    final_filter = [{"bool": {"should": [{"match": {"title": "US"}},
                                              {"match": {"text": "Pooja"}},
                                              ]}}]

What to do next

After you connect to Elasticsearch, you can modify the repository connection data or the settings you provided to fetch the documents.

Also, if you want to fine-tune how your agent uses the search results, see Editing the settings of the search results.