$13 trillion USD is a huge sum of money, difficult to comprehend or compare. But over the next 10 years, that figure will equal the total economic impact that AI will add to the global economy, according to models run by McKinsey.

In financial services, AI is the future, one observer writes, after evaluating a new study conducted by Deloitte. For instance, AI models can incorporate big data—more types of data from multiple sources—and use it to make predictions about who should receive a loan or insurance policy. The Deloitte study noted that 70 percent of financial services firms responding were already using AI models in production.

Any firm using AI must revisit its approach to model risk management. The reason is that AI models are evolving faster than the rules-based models that were standard previously. If AI models perform inadequately, major operational losses can result quickly.

Relevance of SR-11–7 and other regulations

The Federal Reserve Board and the Office of Comptroller of the Currency issued a joint regulation for model risk management called “Supervisory Guidance on Model Risk Management” (SR 11–7) in 2011. This regulation has become the key guideline for financial institutions around the world that need to manage model risk. It initially targeted large institutions with more than $50 billion in assets.

Additional regulations have focused on smaller institutions. The Federal Deposit Insurance Commission (FDIC) announced its adoption of SR 11–7 in 2017 (FIL-22–2017) and reduced the minimum threshold for compliance to banks with only $1 billion in assets.

Issues when using AI models

With guidance such as SR 11-7 in mind, financial organizations face unique challenges when using AI models across the model lifecycle. These include:

Bias: If AI models were trained on data containing unwanted biases, they can result in biased decisions. This can be a major issue if the bias is not detected and mitigated. Loan models reflecting bias have resulted in minority groups receiving higher mortgage rates, for instance, and the organizations that used the models received substantial fines.

Explainability:  Traditional statistical models are simpler to interpret and explain than today’s AI models. AI can be a black box, making it difficult for an organization to explain to internal users or customers why a particular loan application or underwriting decision was approved or denied. In regulated financial industries especially, AI model decisions need to be transparent.

Model drift: AI models can drift in accuracy within days once they are put into production, because production data inevitably differs from training data. This can heighten risk and affect business KPIs. Model drift must be detected and mitigated.

Increased documentation and governance: Organizations can change and adjust AI models more often than they do rules-based models. This is because AI technologies such as AutoAI are being improved and advanced quickly. To assist with governance, it’s critical to have automatic documentation of changes in models and data in place.

What does IBM Watson OpenScale do?

IBM Watson OpenScale was created to help organizations address challenges such as the ones described above. OpenScale helps validate and monitor AI models to enhance compliance and assist in mitigating business risk.

For instance, Watson OpenScale can detect and mitigate AI model bias at runtime. See how in this short credit risk scenario video.

OpenScale can also explain AI outcomes in business-friendly language during runtime. It automatically detects the factors contributing to the model’s decision and shows these graphically. And it uses proprietary technology from IBM Research to provide a “contrastive explanation.” This term refers to a description of the minimum changes that would flip a transaction’s outcome. For example, a loan application might be changed from “risk” to “no risk” if the applicant increased a savings account balance from $500 to $1,000. This type of explanation can be helpful not only to customers, but also to internal stakeholders as they evaluate their AI models. See more about making AI explainable in this short video.

Watson OpenScale also can detect and correct AI model accuracy drift. When it exceeds an acceptable threshold during production, OpenScale generates an alert and identifies the transactions that are causing drift. They can be used to re-train the model and improve its accuracy. Learn more in this short video on mitigating drift.

AI model risk management: Join the Watson OpenScale private beta program

Financial services organizations have additional challenges in AI model risk management, and the Watson OpenScale product team is developing new features to help in this area. Would you like to test these features and offer feedback? Register here for the early access program today.

Learn more about Watson OpenScale here.

Accelerate your journey to AI.

Was this article helpful?

More from Analytics

How the Recording Academy uses IBM watsonx to enhance the fan experience at the GRAMMYs®

3 min read - Through the GRAMMYs®, the Recording Academy® seeks to recognize excellence in the recording arts and sciences and ensure that music remains an indelible part of our culture. When the world’s top recording stars cross the red carpet at the 66th Annual GRAMMY Awards, IBM will be there once again. This year, the business challenge facing the GRAMMYs paralleled those of other iconic cultural sports and entertainment events: in today’s highly fragmented media landscape, creating cultural impact means driving captivating content…

How data stores and governance impact your AI initiatives

6 min read - Organizations with a firm grasp on how, where, and when to use artificial intelligence (AI) can take advantage of any number of AI-based capabilities such as: Content generation Task automation Code creation Large-scale classification Summarization of dense and/or complex documents Information extraction IT security optimization Be it healthcare, hospitality, finance, or manufacturing, the beneficial use cases of AI are virtually limitless in every industry. But the implementation of AI is only one piece of the puzzle. The tasks behind efficient,…

IBM and ESPN use AI models built with watsonx to transform fantasy football data into insight

4 min read - If you play fantasy football, you are no stranger to data-driven decision-making. Every week during football season, an estimated 60 million Americans pore over player statistics, point projections and trade proposals, looking for those elusive insights to guide their roster decisions and lead them to victory. But numbers only tell half the story. For the past seven years, ESPN has worked closely with IBM to help tell the whole tale. And this year, ESPN Fantasy Football is using AI models…

Data science vs data analytics: Unpacking the differences

5 min read - Though you may encounter the terms “data science” and “data analytics” being used interchangeably in conversations or online, they refer to two distinctly different concepts. Data science is an area of expertise that combines many disciplines such as mathematics, computer science, software engineering and statistics. It focuses on data collection and management of large-scale structured and unstructured data for various academic and business applications. Meanwhile, data analytics is the act of examining datasets to extract value and find answers to…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters