Configuring default chunk settings for search queries

You can set the default number of document chunks that Content Assistant sends to the LLM when processing queries.

Before you begin

Make sure that you have one of the following:
  • For Content Platform Engine 5.7.0 deployments, you must have Content Platform Engine 5.7.0 IF5 or later.
  • For deployments earlier than Content Platform Engine 5.7.0, you must have the latest version of Content Assistant.

About this task

When a natural language query is submitted to IBM® Content Assistant, the system retrieves relevant document chunks from the vector database and sends them to the LLM. You can configure the default number of chunks sent to the LLM for single document queries and multiple document queries. You can adjust the default chunk numbers by editing the GenAI configuration object and specifying chunk numbers as advanced configuration options.

Procedure

  1. In the Administration Console for Content Platform Engine (ACCE), access the Gen AI Configuration object for the object store where you want to configure default chunk numbers for Content Assistant.
    Tip: Starting with Content Platform Engine 26.0, you can open the Gen AI Configuration object for the selected object store directly from Administrative > Content Assistant > Gen AI Configuration.
    1. Go to the object store where you want to configure default chunk numbers for Content Assistant.
    2. In the left pane, click Search.
    3. Click New Object Store Search.
    4. In Simple View, specify the Class as Gen AI Configuration, and click Run.
      The search results display a single Gen AI Configuration object.
    5. From the search results, click the object ID to access the properties for the Gen AI Configuration object.
  2. In the Properties tab, specify the default chunk numbers in the Gen AI Advanced Options field.

    Specify the chunk numbers as JSON values. The value must be between 1 and 32 for each option. You can configure:

    • DocumentQueryDefaultNumberOfChunks - Sets the default number of chunks for single document queries
    • VectorQueryDefaultNumberOfChunks - Sets the default number of chunks for multiple document queries and repository-wide searches
    For example, to set 11 chunks for single document queries and 12 chunks for multiple document queries, enter {"DocumentQueryDefaultNumberOfChunks":11,"VectorQueryDefaultNumberOfChunks":12}.
    Note: API callers can override these defaults by specifying their own chunk values in API calls.
  3. Save your changes.