Area under ROC evaluation metric

The area under ROC metric measures how well your model identifies differences between classes.

Metric details

Area under receiving-operating characteristic (ROC) is a quality evaluation metric that measures the quality of the performance of binary classification machine learning models in watsonx.governance.

Scope

The area under ROC metric evaluates machine learning models only.

  • Types of AI assets: Machine learning models

  • Machine learning problem type: Binary classification

Scores and values

The Area under ROC metric score indicates how well the model identifies differences between classes. Higher scores indicate better model performance with identifying classes.

  • Range of values: 0.0-1.0
  • Best possible score: 1.0
  • Chart values: Last value in the timeframe

A score of 0.5 suggests random guessing, while a score of 1.0 represents perfect classification.

Settings

Default threshold: Lower limit = 80%

Evaluation process

The area under ROC metric is calculated by plotting the True positive rate (TPR) against the False positive rate (FPR) for different threshold values. For each threshold, a confusion matrix is generated that specifies classes of true positives, false positives, true negatives, and false negatives.

The TPR and FPR are calculated with these classes and plotted on a graph to create the ROC curve. The area under this curve is calculated to generate the metric score.

Parent topic: Evaluation metrics