IBM enhances the capabilities of watsonx.governance with the new Model Risk Evaluation Engine
We're excited to announce the Model Risk Evaluation Engine, a new tool in watsonx.governance developed with IBM Research, that can measure risks of foundation models by computing metrics related to risk dimensions from the AI Risk Atlas. As part of a comprehensive model onboarding process, it enables the comparison of risk metrics across different foundation models, helping you identify the most suitable foundation models for deployment within your organization, in line with your organization’s specific risk tolerance.
As companies continue to scale their generative AI deployments, it is increasingly important for them to develop a better understanding of the risks associated with the underlying foundation models, including prompt injection, toxic output, jailbreaking and hallucination.
Organizations have a lot of choice when it comes to selecting which generative AI models to use in their company. Making an informed decision is important to avoid situations when generative models perform poorly (for example a “rogue” customer-facing chatbot giving incorrect or harmful guidance). These situations can have a huge reputational impact that can be hard for an organization to recover from. Thus, having objective quantitative risk data to prevent these types of situations should be part of an organization’s model onboarding process.
The onboarding process for generative AI models has 3 stages:
- Understand the general risks of generative AI.
- Identify the risks that are applicable to a particular AI model (or use case).
- Evaluate the identified risks.
Onboarding a library of risks is the first step to understanding what risks may be applicable. The AI Risk Atlas from IBM is a great resource for understanding the risks associated with the use of generative AI and machine learning models. The risks are also integrated directly into the Governance Console of watsonx.governance, and are available out of the box. The library of risks can also be supplemented with an organization’s own inventory of risks if desired. The risks can be linked to AI Use Cases and Models using the out-of-box risk identification assessments (AI Use Case, Model Onboarding and Use Case + Model combined).
Understanding the risks that may be applicable is a great first step, but it is equally important to have effective methods for identifying, measuring and mitigating these risks.
watsonx.governance ships with 3 risk identification assessments:
- AI use case risk identification: used to identify risks that are specific to the use case being proposed, and are not model-specific. For example, prompt injection, IP information in prompt and exposing personal information.
- AI model onboarding risk identification: used to identify risks that are specific to the model being assessed. For example, data bias, uncertain data provenance, lack of training data transparency and reidentification.
- Use case + model risk identification: used to identify an additional set of risks that could emerge from a specific combination of a use case and a model. For example, model usage rights restrictions, hallucination and unexplainable output.
These assessments are used in determining which risks from the Risk Atlas are applicable to the model and/or use case being onboarded. In the watsonx.governance Governance Console, there is a workflow for foundation model onboarding that includes the risk identification questionnaire assessment mentioned above.
After being identified, the applicable risks should be individually examined using a Risk and Control Self-Assessment (RCSA) to determine inherent and residual risk. This will produce a risk profile for the model that can inform which type of uses an organization would be willing to approve for the model, like RAG, classification or summarization.
To better inform the RCSA process, some quantitative evaluation can be performed to gain a deeper understanding of the risk of a certain model and how it compares to similar models. Companies are also empowered to evaluate the risks of any model they develop or enhance (for example through fine-tuning).
The Model Risk Evaluation Engine, now part of watsonx.governance, aids with the quantitative risk assessment of foundation models. It computes metrics that are related to a defined set of risk dimensions from the AI Risk Atlas. By computing these metrics for a wide selection of foundation models, companies are empowered to choose the models that meet their business objectives while also aligning to their risk appetite.
The Model Risk Evaluation Engine supports the evaluation of large language models from IBM watsonx.ai as well as any external large language models. The completed results of the evaluation engine can be saved to the Governance Console of watsonx.governance or exported as a PDF report.
Model Risk Evaluation Engine helps to accomplish the following tasks:
- Compute metrics using watsonx.ai as the inference engine
- Compute risk metrics for foundation models in watsonx.ai
- Compute risk metrics for external foundation models
- Store computed metrics in the Governance Console (OpenPages)
- Retrieve computed metrics from the Governance Console (OpenPages)
- Add your own risks and datasets
- Generate a PDF report of computed metrics
- Implement your own scoring function for any model perform evaluations (for example, deterministic function, or LLM-as-a-judge)
- Display the metrics in a notebook cell, in a table or chart format
Once all of this data flows back into the Governance Console, it can be used to inform the risk assessment step of the foundation model onboarding workflow described above.
watsonx.governance users can access Model Risk Evaluation Engine by running the following command:
Our sample notebook contains instructions to give it a try for yourself. The Model Risk Evaluation Engine documentation page also contains more information.
If you’d like your organization to identify, measure and mitigate generative AI risk effectively, an end-to-end AI governance solution like watsonx.governance is crucial. Try it out for yourself or set up a time for us to discuss with an IBM expert today.