Sentence similarity evaluation metric

The sentence similarity metric captures semantic information from sentence embeddings to measure the similarity between texts.

Metric details

Sentence similarity is a generative AI quality evaluation metric that measures how well generative AI assets perform tasks.

Scope

The sentence similarity metric evaluates generative AI assets only.

  • Types of AI assets: Prompt templates
  • Generative AI tasks: Text summarization
  • Supported languages: Arabic (ar), Danish (da), English (en), French (fr), German (de), Italian (it), Japanese (ja), Korean (ko), Portuguese (pt), Spanish (es).

Scores and values

The sentence similarity metric score indicate the similarity between texts. Higher scores indicate that the texts are more similar.

  • Range of values: 0.0-1.0
  • Best possible score: 1.0

Settings

  • Thresholds:
    • Lower limit: 0.8
    • Upper limit: 1