Chapter 01Chapter 02Chapter 03Chapter 04Chapter 05Chapter 06

No relevant matches. Try broadening your query.

AI models lose value if their results can’t be explained

Chapter 05
5 min read

AI models are gaining more widespread adoption, demonstrating impressive accuracy across various industries. However, it’s not enough to simply demonstrate a model’s accuracy; it must be explainable and provable. Unfortunately, even the most well-trained AIs — free of bias and drift — often are not easily understood by the people who interact with and are affected by them.

Woman watching moving data
91% of organizations say their ability to explain how their AI made a decision is critical. 1

Explainability is crucial because it provides insight into the AI’s decision-making process. If you don’t understand how an AI model arrived at a result, you can’t fully trust the model itself. Not only that, but from a regulatory standpoint, the inability to explain how the model draws its conclusions means those conclusions aren’t likely to be compliant.

But what does this all mean for your business? As with a medical diagnosis or loan application, many variables contribute to a conclusion, and all data points must be considered to generate a trustworthy result. Without explainable AI, you can’t be confident that the AI models being put into production are completely reliable. AI explainability is crucial to help organizations develop and use AI responsibly and reliably.


of business leaders believe that customers will demand more explainability from AI in the next three years.2

As AI continues to become more complex, it’s increasingly difficult to understand and retrace how an algorithm came to a result. The whole AI calculation process is often considered to be a “black box” that is difficult to interpret. This means it’s even more crucial for businesses to monitor and manage models so they can understand and measure the impact of using certain algorithms.

When it comes to explainability, there are two components: (1) a business needs to be able to explain the decision made by the AI and (2) the business needs to be able to explain the history of the project. What was the data’s path? What was the original intent of the AI model?

All these things need to be explained to provide transparency across the entire AI model. To have ethical AI that isn’t causing inequalities, companies need to understand who is running their AI systems, what data was used, and what went into their algorithms’ recommendations. To understand the behavior of a model, you need to understand what the model is doing.


of AI high performers track AI-model performance and explanations to ensure that outcomes and/or models improve over time.3

Explainability can also help developers ensure that a system is working as expected and complying with regulatory requirements. GDPR standards have had a profound effect on how customer information is gathered and used, with stiff penalties for businesses found to be in violation. And new regulations are coming.

Without explainability, AI simply can’t be implemented responsibly at scale. Businesses need to embed ethics principles into their AI applications and processes by building an AI system that is grounded in transparency.4

What percentage of businesses have questions or concerns about explaining AI outcomes and ensuring it is fair?
Five individuals analyzing data

Of businesses have concerns when it comes to AI model bias, fairness, and explainability.

1 Trustworthy AI, IBM, June 2021.
3 The State of AI in 2020, McKinsey & Company, November 2020.
4 Explainable AI, IBM, March 2021.