Text extraction parameters

Configuring the key-value pair text extraction pipeline with the semantic_config parameter

You can use various common fields of the semantic_config parameter in the REST API request body to configure the key-value pair extraction pipeline. For details, see Configuring the key-value pair processing pipeline.

In addition, you can configure the following capabilities specific to the key-value pair extraction pipeline in the semantic_config parameter:

• Extraction method
• Pre-defined and custom schema usage
• Grounding mode

Setting the extraction method with the enable_schema_kvp and enable_generic_kvp fields

Based on the contents of your input document, you can process key-value pair data with one of the following methods:

Schema-based key-value pair extraction

The schema-based process targets specific fields in documents by using pre-defined or custom schemas for common document types like invoices, utility bills, passports, and more. Every page is classified into one of the supported schema types. Based on the classification, you can extract text into the key-value pair format defined in the schema for the specific document type. By classifying the document and targeting domain-specific data, this method increases accuracy for known document types without requiring dedicated model training.

Generic key-value pair extraction

The generic extraction process identifies and extracts content that can be represented as key-value pairs from a document. This method is useful for extracting labeled information when you do not have domain-specific knowledge of the document. Generic extraction works best for pages that do not fit into one of the pre-defined templates.

By default, both the generic and schema-based methods are used to extract text in the generic_with_semantic mode. You can choose to use one or both of the extraction methods in the same API request to accomplish the following tasks:

• Extract a targeted set of fields with a schema by using the schema-based extraction method
• Perform a broad sweep of the document to extract all generic data by using the generic extraction method

To use the schema-based extraction method, set the following semantic_config parameter in the text extraction request:

"semantic_config": {
  "enable_schema_kvp": true
}

To use the generic extraction method, set the following semantic_config parameter in the text extraction request as follows:

"semantic_config": {
  "enable_generic_kvp": true
}

Defining pre-defined and custom schema interaction with the schemas_merge_strategy and force_schema_name fields

You can specify how a combination of pre-defined and custom schemas are used to process key-value pair data in your document.

You can use common semantic_config settings such as the schemas_merge_strategy field to control how pre-defined and custom schemas are used together. For details, see Controlling how pre-defined and custom schemas interact.

However, if your document can be classified into one of the pre-defined document types, such as a receipt, but contains some data in a customized format specific to your business needs, you can define your own custom schema for the document. You can then manually override the pre-defined schema and use the custom receipt schema directly by setting the force_schema_name parameter in your API request as follows:

"semantic_config": {
  "enable_schema_kvp": false,
  "enable_schema_kvp": true,
  "force_schema_name": "Receipt",  # Force the document to be processed as a receipt
  "schemas": [ {
      "document_type": "Receipt",
      "document_description": "A receipt issued for a purchase at ABC store.",
      "fields": {
        "receipt_number": {
          "default": "",
          "example": "R-20241027-ABC",
          "description": "Unique identifier on the receipt."
        },
        "customer_name": {
          "default": "",
          "example": "John Smith",
          "description": "Full name of the customer or payee."
        },
        "date_of_transaction": {
          "default": "",
          "example": "2023-01-01",
          "description": "Date when the purchase or payment occurred."
        },
        "total_paid": {
          "default": "",
          "example": "8.64",
          "description": "Final amount paid by the customer."
        },
        "payment_method": {
          "default": "",
          "example": "Credit Card",
          "description": "How payment was made, such as cash, card, check, etc.)."
        },
      }
  } ]
}

The following table describes the difference in the methods you can use to process your documents with custom schemas:

Including key-value pair location with the grounding_mode field

A grounded key-value pair has both the key and value associated with a physical location in the document. The physical location is represented as a bounding box in the extracted output.

Grounding information for extracted structured and semi-structured data provides the following capabilities:

Enhanced verification

When a reviewer needs to validate or correct the extracted data, bounding boxes provide visual context, and make it easy to locate the key and value in the original document.

Traceability and auditability

Grounding allows end-users or downstream systems to trace each extracted piece of data back to its source location. Traceability supports audits and compliance checks, and facilitates dispute resolution processes.

Integration with UI workflows

Applications like document viewers or annotation tools rely on bounding boxes to highlight or allow interaction with extracted fields. Grounding data connects raw data with user interface experiences.

The following table provides details about the different modes you can use in the grounding_mode setting in the semantic_config parameter to set the precision of grounding information to include with extracted key-value pair data: