Contains string evaluation metric

The contains string metric measures whether each row in the prediction contains the specified string.

Metric details

Contains string is a content validation metric that uses string-based functions to analyze and validate generated LLM output text. The metric is available only when you use the Python SDK to calculate evaluation metrics.

Scope

The contains string metric evaluates generative AI assets only.

  • Types of AI assets: Prompt templates
  • Generative AI tasks:
    • Text summarization
    • Content generation
    • Question answering
    • Entity extraction
    • Retrieval augmented generation (RAG)
  • Supported languages: English

Scores and values

The contains string metric score indicates whether the prediction rows contain the specified string.

  • Range of values: 0.0-1.0
  • Ratios:
    • At 0: The prediction rows do not contain the specified string.
    • At 1: The prediction rows contain the specified string.

Settings

  • Thresholds:
    • Lower limit: 0
    • Upper limit: 1

Parent topic: Evaluation metrics