Fuzzy match evaluation metric

The fuzzy match evaluation metric measures the similarity between generated predictions and keywords in reference answers.

Metric details

Fuzzy match is a content validation metric that uses string-based functions to analyze and validate generated LLM output text. The metric is available only when you use the Python SDK to calculate evaluation metrics.

Scope

The fuzzy match metric evaluates generative AI assets only.

  • Types of AI assets: Prompt templates
  • Generative AI tasks:
    • Text summarization
    • Content generation
    • Question answering
    • Entity extraction
    • Retrieval augmented generation (RAG)
  • Supported languages: English

Scores and values

The fuzzy match metric score indicates the level of similarity between generated predictions and reference keywords.

  • Ratios:
    • At 0: The prediction does not match the keyword.
    • At 1: The prediction matches the keyword.

Settings

  • Thresholds:
    • Lower limit: 0
    • Upper limit: 1

Parent topic: Evaluation metrics