Before AI can help your business reach new levels of productivity, you need to be able to trust what it’s doing.

While generative AI has the potential to unlock tremendous productivity and economic value, it comes with new complexities and increased risks not previously seen with predictive machine learning (ML). This ranges from the origin of underlying training data to the potential of AI to perpetuate bias to a lack of explainable outputs. Businesses must establish guardrails to manage these risks, embrace transparency, and anticipate addressing compliance with future AI-focused regulation.

IBM has already been working with clients on governing AI and its machine learning governance capabilities were recently named a leader in the IDC MarketScape: Worldwide AI Governance Platforms 2023 Vendor Assessment. As part of our commitment to trust, and open innovation, IBM also announced today it has partnered with Meta and over 50 founding members to form the AI Alliance with technology companies around the world to promote the safe and responsible use of AI.

IDC: “The AI governance platform from IBM is a comprehensive solution for enabling responsible and transparent AI practices throughout the model life cycle.”

Introducing watsonx.governance

Watsonx.governance is designed to be a one-stop-shop for businesses navigating how to deploy and manage both LLM and ML models. It provides tools to help them mitigate risks and accelerate responsible, transparent and explainable generative AI and machine learning (ML) workflows. Watsonx.governance is part of the watsonx AI and data platform, which also includes watsonx.ai, an enterprise studio for AI builders and watsonx.data, a fit-for-purpose data store based on an open lakehouse architecture. Built on a strong foundation of IBM’s AI governance technologies, watsonx.governance can help you operationalize AI with confidence in three main ways:

• Compliance: Manage AI to address internal policies, industry standards and help prepare for upcoming regulations and policies worldwide—a “nutrition label” for AI.
• Risk management: Proactively detect and mitigate risks monitoring for fairness, bias, drift and new LLM metrics.
• Lifecycle governance: Manage, monitor and govern AI models from IBM, open source communities and other model providers.

Exploring the capabilities of watsonx.governance for LLMs

There are three main capabilities available in watsonx.governance for LLMs that work together to help businesses address compliance, risk management and lifecycle governance.

Address compliance with tracking and transparency

Preparing for growing and changing AI industry standards should include documentation. Automating the capture and documentation of model facts is critical for establishing transparent model processes with explainable results throughout the model life cycle. There is a growing need for documentation to support audits and regulatory inquiries and to provide key performance metrics to key stakeholders. Watsonx.governance uses factsheets to automatically log and monitor model facts. At IBM we refer to them as a “nutritional label” for models as it provides a repository of all relevant information about the model, hyperparameters, metrics and model evaluations and stores them as model metadata. These documents facilitate a comprehensive performance and risk management view across the model lifecycle and serve as a record of the development activity and performance metrics.

Factsheets contain the identified model, prompt template, model parameters and other pertinent information that the data scientist or model validator chooses to include. They are customizable, easily accessible by stakeholders and can be printed or downloaded to be sent as an attachment for those without access to the application. Automated factsheets help minimize the time and cost of manually supporting audits, providing key performance metrics and responding to regulatory requests.

Manage risk with model evaluation and documentation

The proactive detection and mitigation of risks is key in avoiding inaccurate and biased model outcomes. Manual evaluation and monitoring can lead to human errors, and delays in model deployment and inaccurate model outcomes can lead to audits, fines, lost revenue and damage to an organization’s reputation. Automating risk management with alerts when metrics are out of range is key in driving responsible and ethical AI.

Evaluation metrics in watsonx.governance are available for several use cases including text summarization, text classification, language translation, content generation, retrieval augmented generation (RAG) and Q&A. Prompt performance can be checked periodically to help make sure they are performing accurately and not producing potentially harmful or inappropriate content.

Monitor models with lifecycle management

Lifecycle management involves closely monitoring the behavior of models in production. Watsonx.governance enables data scientists to proactively identify and remediate issues regarding drift and accuracy. Lack of continuous, automated monitoring can result in undetected change in the performance of the model over time. 

Watsonx.governance continuously monitors AI performance metrics to detect issues related to drift, quality, and bias. Preset thresholds monitor both the inputs and the outputs for generative AI and alert when thresholds have been breached for toxic language, hate speech, abusive language and profanity. Watsonx.governance monitors for data size, latency and throughput change.

With watsonx.governance you can govern both LLMs and machine learning (ML) on one platform, watsonx. While the focus of AI in the press and media has been on LLMs, machine learning models are being actively used in areas like customer service, fraud detection, diagnosis and treatment in healthcare. Watsonx.governance governs machine learning models from any vendor and is deployed in the cloud and on-premises. Capabilities for ML models include monitoring for fairness, drift and quality, automate “de-bias” tools and what-if-analysis. 

How you can get started today

In the age of AI for business, watsonx.governance drives the ability to direct, manage and monitor the AI activities for your entire organization with enterprise level-rigor and oversight into how both your ML and generative AI models are created and deployed.

Test watsonx.governance for yourself.

Disclaimer: IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract.The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.