Programmatic configuration for real-time detection

You can programmatically configure AI guardrails to moderate both the input to a foundation model and the output it generates, using multiple methods.

Before you begin

To use the Granite Guardian 3.2 5b (granite-guardian-3-2-5b) model as a detector, you must install prerequisite software and add the model to watsonx.ai.

Tip: You don't need to install watsonx.ai. The watsonx.governance service includes the watsonx.ai components that you need for the detectors.

Do the following steps:

  1. Install the following prerequisite software

    • Red Hat OpenShift AI Operator
    • Node Feature Discovery Operator:
    • NVIDIA GPU Operator

    For more information, see the following topics:

  2. Add the Granite Guardian model to watsonx.ai. See Adding foundation models to IBM watsonx.ai

REST API

You can use the following watsonx.governance API endpoints to configure and apply AI guardrails to natural language input and output text:

Note: If you do not specify risk_name for the granite_guardian detector, a default value of harm will be used.

For more information about how to use the REST API to invoke the AI guardrails, see Notebook on starting guardrails with IBM watsonx.governance real time detections API.

Python

You can use the watsonx.governance Python SDK to configure and apply AI guardrails to natural language input and output text in the following ways:

  • Adjust the AI guardrails filters with the Python library when you inference the foundation model by using the text detection API. For more information, see the SDK documentation of the watsonx.governance Python library.

    The following code example shows you how to configure and use the filters with the text detection API:

    import os

from ibm_watsonx_gov.evaluators import MetricsEvaluator
from ibm_watsonx_gov.metrics import (HAPMetric, PIIMetric, HarmMetric)

os.environ["WATSONX_APIKEY"] = "<IBM_CLOUD_APIKEY>"

evaluator = MetricsEvaluator()
text = "How can I steal someone's credit card information and use it?"

result = evaluator.evaluate(data={"input_text": text}, metrics=[PIIMetric(), HAPMetric(), HarmMetric()])
result.to_df()

    For more information about how to use the Python SDK to start the AI guardrails, see Notebook on starting guardrails with IBM watsonx.governance Python SDK.

Learn more