An end-to-end toolkit for AI governance across to manage risk, compliance and the entire AI lifecycle
Gen AI has the potential to boost productivity and unlock trillions in economic value. However, these models come with new complexities and risks. Regardless of whether you are using gen AI models or traditional ML models, every model needs governance.
IBM® watsonx.governance™ toolkit for AI governance provides users with model choice and flexibility. This toolkit can:
- govern gen AI models that are built on IBM® watsonx.ai™ and models that are developed on third-party platforms, including Amazon Bedrock, Microsoft Azure and OpenAI
- manage machine learning (ML) models built in third-party tools
- let clients choose between cloud or on-premises deployments for both model types
To govern an enterprise's gen AI effectively, it is essential to monitor and mitigate the newly amplified risks originating from models, users, data sets and regulations. Watsonx.governance offers 3 key capabilities for achieving this.
Watsonx.governance uses factsheets to automatically log and monitor model facts. At IBM, we refer to them as 'nutritional labels' for models because they contain a repository of all relevant model information. These documents facilitate a comprehensive view of performance and risk management throughout the model's lifecycle, serving as a record of development activities and performance metrics. Users can download factsheets or send them as attachments to stakeholders or to support audits.
Evaluation metrics are available for a range of use cases, including text summarization, text classification, language translation, content generation, retrieval augmented generation (RAG) and question and answer. Prompt performance can be checked throughout the AI lifecycle to help ensure accurate performance and to prevent the generation of potentially harmful or inappropriate content. Factsheets are used to document these evaluation metrics.
Performance metrics are monitored to avoid issues that are related to drift, quality and safety. Preset thresholds monitor both the inputs and the outputs of the gen AI model, and they provide alerts when toxic language, hate speech, abusive language or profanity is detected in the model's inputs and outputs. Watsonx.governance monitors for data size, latency and changes in throughput.
Footnotes
IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. See Pricing for more detail. Unless otherwise specified under Software pricing, all features, capabilities, and potential updates refer exclusively to SaaS. IBM makes no representation that SaaS and software features and capabilities will be the same.