Governing assets with watsonx.governance

Use watsonx.governance to accelerate responsible, transparent, and explainable AI workflows with an AI governance solution that provides end-to-end monitoring for machine learning and generative AI models. Monitor your foundation model and machine learning assets from request to production. Collect facts about models that are built with IBM tools or third-party providers in a single dashboard to aid in meeting compliance and governance goals.

Develop a comprehensive governance solution

Using watsonx.governance, you can extend the best practices of AI governance from predictive machine learning models to generative AI while monitoring and mitigating the risks associated with models, users, and data sets. The benefits of this approach include:

  • Responsible AI: extend the practices of responsible AI from governing predictive machine learning models to the use of generative AI with any foundation or model provider.
  • Explainability: Use automation to improve transparency and explainability for tracked models. Use tools for detecting and mitigating risks that are associated with AI.
  • Transparent and regulatory policies: Mitigate AI risks by tracking the end-to-end AI lifecycle to aid compliance with internal policies and external regulations for enterprise-wide AI solutions.

Watsonx.governance builds on the IBM AI Governance solution by adding governance for prompt templates for foundation models.

Components of watsonx.governance

Watsonx.governance includes these capabilities for addressing your governance needs in an integrated solution:

Evaluate and monitor of AI assets

Evaluation and monitoring of AI assets extends the capabilities of Watson OpenScale for configuring monitors that evaluate your deployed assets against thresholds you specify. For example, you can configure thresholds that alert you when predictive machine models perform under a specified threshold for fairness in monitored outcomes, or drift from accuracy. You can also monitor prompt templates for dimensions to measure generative AI interactions. A Model Health monitor provides real-time performance tracking for deployed models.

Monitoring a deployment for generative AI quality

Track facts for governance in AI use cases

The integrated AI Factsheets solution collects the metadata for machine learning models and prompt templates for foundation models that you explicitly track. Develop AI use cases to gather all of the information for managing an AI asset from the request phase through development and into production. Manage multiple versions of a model, or use different approaches to solving a business problem within a use case. Factsheets display information about tracked assets, including creation information, data that is used to train or prompt a model, and where the asset is in the lifecycle. View high-level information for all of the use cases you can access or view the details of a particular use case, all in service of meeting policy and compliance goals.

Track assets through the AI lifecycle

Manage governance and compliance from the Governance console

Building on the model risk governance capabilities of IBM OpenPages Model Risk Governance, you can collect metadata about prompt templates for foundation models and machine learning models to help you achieve your governance goals. Use Governance console to deliver a comprehensive view of governance activity across an organization

Viewing activity for governed assets from the Governance console

Extend governance with watsonx.ai

To create an end-to-end experience for developing assets and then adding them to governance, use watsonx.ai with watsonx.governance. Watsonx.ai extends the Watson Studio and Watson Machine Learning services to work with foundation models, including capabilities for saving prompt templates for a curated collection of large language model assets.

In addition to working with foundation models curated by IBM, you can use watsonx.governance APIs to evaluate prompts for external large language models. View annotated sample notebooks for details on connecting to generative AI models hosted by AWS Bedrock, Google Vertex AI, and Azure OpenAI. Watsonx.governance supports AI everywhere to meet the needs of your enterprise.

Governance in action

Your governance strategy is tailored for the needs of your organization. Governance is a collaborative process, involving users from various roles and areas of expertise.

The following graphic depicts a typical governance flow, starting with defining an AI use case to solve a business problem and requesting an AI asset, such as a model or prompt template, to solve the problem. The figure shows the various roles that might be involved in the flow, starting with a model owner who defines the problem, then moving from the developer who builds the asset, to a validator who tests it. In the next step, a risk officer might review and approve the solution, hand it off to an ML Ops engineer to deploy it, and then deliver it to an App developer who can monitor the asset in production. Your approach might combine some of these roles.

The watsonx.governance flow

A governance scenario

A typical governance scenario might include this sequence:

  1. A business user identifies a need for an AI solution and creates an AI use case to request a new model.
  2. When the request is saved, the AI use case is created in the inventory, and the tracking begins. Initially, the use case is in the Draft state because there are no assets to accompany the request. Optionally, an approval process can be implemented to automate reviewing and approving the use case before assigning it to a data scientist.
  3. When a data scientist creates a model or prompt template for the use case, they associate the asset with an approach in the use case. An approach represents one facet of the solution that is tracked in the use case. A use case can include multiple approaches. The details for the asset, such as training data and creation details are captured in a factsheet and stored in the use case.
  4. As the asset advances in the lifecycle, the use case and the associated factsheets capture all updates, including deployments and evaluation results. Collaborators can generate reports from AI uses cases to use for compliance goals or archiving. From the Governance console, users can also use report views to verify regulatory compliance or save the report views for archival purposes.
  5. Validators and other stakeholders can review individual AI use cases or view enterprise governance activity from the Governance console to ensure compliance with corporate protocols.

Next steps

  • Plan your governance strategy
  • Review the Risk Atlas to learn about the potential risks of working with AI models. The Risk Atlas provides a guide to understanding some of the risks of working with AI models, including generative AI, foundation models, and machine learning models. In addition to describing potential risks, it provides real-world context. It is intended as an educational resource and is not meant as a prescriptive tool.
  • Follow the tutorial Quick start: Evaluate and track a prompt template to evaluate and track a sample prompt template.

Parent topic: AI governance