Contains_JSON evaluation metric

The contains_JSON metric measures if the rows in the prediction contain JSON syntax.

Metric details

Contains_JSON is a content validation metric that uses string-based functions to analyze and validate generated LLM output text. The metric is available only when you use the Python SDK to calculate evaluation metrics.

Scope

The contains_JSON metric evaluates generative AI assets only.

  • Types of AI assets: Prompt templates
  • Generative AI tasks:
    • Text summarization
    • Content generation
    • Question answering
    • Entity extraction
    • Retrieval augmented generation (RAG)
  • Supported languages: English

Scores and values

The contains_JSON metric score indicates whether the prediction rows contain JSON syntax.

  • Range of values: 0.0-1.0
  • Ratios:
    • At 0: The prediction rows do not contain valid JSON syntax.
    • At 1: The prediction rows contain valid JSON syntax.

Settings

  • Thresholds:
    • Lower limit: 0
    • Upper limit: 1

Parent topic: Evaluation metrics