AI misinformation: Here's how to reduce your company’s exposure and risk

Accurate information is the lifeblood of modern companies. They rely on it to make key business decisions on everything from supply chain management to product marketing.

While artificial intelligence (AI) can improve such data-driven decision-making, it can also hinder it. AI-generated content can be rife with errors, from fake news headlines and terrible legal advice to pizza recipes featuring Elmer’s glue as a key ingredient.¹

Will we ever reach a point where such sticky situations are a thing of the past? As much as AI enthusiasts would love to believe otherwise, the answer is probably not.

Generative AI (gen AI) models, explains IBM’s Matt Candy, will always be vulnerable to inadvertently producing at least some misinformation. “By virtue of the fact that these things are predictive in nature—they’re predicting and guessing what the next word is—you’re always going to have some risk of that,” says Candy, Global Managing Partner for Generative AI at IBM Consulting®.

Candy adds that traditional machine learning (ML) models aren’t immune from producing misinformation, either. “Those models are statistical kinds of machines that, effectively, are trying to predict some sort of outcome,” he says. “So at the end of the day, those models can still predict an incorrect answer or outcome.”

However, the good news is that there are multiple steps that companies can take to reduce the chances that their own AI systems produce and spread misinformation.

If those measures don’t prevent all instances of AI-generated misinformation, companies can also implement safeguards to detect the misinformation before it causes harm.

Before we examine how companies can protect themselves from AI-generated misinformation, it’s important to take a closer look at misinformation and related concepts. These concepts largely predate today’s widespread use of AI in information ecosystems, but the influence of AI on these phenomena is significant.

Misinformation is false information. Some definitions also note that misinformation is not purposely created to deceive others, but rather is caused by errors.

In the case of AI-generated misinformation, examples might include large language models (LLMs) producing inaccurate or nonsensical output, and AI-generated images with unrealistic or inaccurate depictions (such as “real people” with too many arms).

AI hallucinations are a common cause of AI-generated misinformation. AI hallucinations occur when AI algorithms produce outputs that are not based on training data, are incorrectly decoded by the transformer or do not follow any identifiable pattern.

“The way hallucinations happen is that the AI model is trying to make sure that the language is fluent, but it’s also trying to piece together different sources of information,” explains Kush Varshney, an IBM Fellow at IBM Research®. “Even with humans, when we try to do multiple things at the same time, we can mess up. This is also happening with the AI model, it’s losing track of information while trying to make the language fluent, and vice versa.”

Disinformation is sometimes considered a type of misinformation, but it is distinct in that it is fake content created to deceive its audience. Examples include conspiracy theories, and more recently, fabricated audio and visual materials.

Prominent cases of disinformation enabled by bots and other AI tools took place in the lead-up to the 2024 American presidential election. These included a robocall impersonating the voice of then-President and Democratic candidate Joe Biden, and the spread of images conveying false celebrity endorsements of Republican president Donald Trump.^2,3

Tools for detecting AI-generated deepfakes and other deceptive, disinformation content have delivered mixed results, though the latest generation of AI-text detectors has proven more effective than previous iterations.^4,5 Meanwhile, social media platforms such as TikTok and Facebook have started labeling AI-generated content.⁶

Unlike misinformation and disinformation, malinformation is potentially harmful information based on reality and facts. Malinformation is damaging because it is distributed to hurt others, such as people or companies.

For example, sharing confidential information without permission falls under the umbrella of malinformation, a practice that’s been amplified by AI. Scammers can use generative AI tools to craft sophisticated and effective phishing emails that can help them obtain and spread confidential information.

While it might be impossible to ensure that all AI outputs are completely error-free, there are steps companies can take to significantly reduce the likelihood that their AI systems produce inaccurate or wholly fabricated information.

• Ensuring data quality
• Deploying retrieval augmented generation (RAG)
• Using smaller generative AI models

High-quality data is critical for AI model performance. Models should be trained on diverse, balanced and well-structured data to minimize the chances of both bias and hallucinations. Tech companies and AI developers can improve the quality of training data by using data preparation and data filtering tools to remove low-quality data and hateful content, including malinformation.

One of the most popular tools for reducing the likelihood of AI hallucination is retrieval augmented generation (RAG). RAG is an architecture that connects generative AI models to external data sources, such as a company’s organizational data, academic journals and specialized datasets. By accessing such information, AI chatbots and other tools can produce more accurate, domain-specific content.

While consumer-facing LLM applications in OpenAI’s ChatGPT have captured the public’s attention, businesses often find smaller, more specialized AI models to better suit their needs, while also being less vulnerable to hallucinations.

“You’ve got these big frontier models trained on as much data as they can get,” Candy says. “But if you think about most enterprise use cases, you don’t need a model that’s trained on the entire works of Shakespeare, Reddit and every other piece of publicly available data.”

Because smaller models have narrower context windows and use fewer parameters, their risk of hallucination declines. “There’s less of a possibility of things getting mixed up,” adds Varshney.

When avoiding harm caused by AI-generated misinformation, measures to mitigate hallucinations are only part of the puzzle. Companies should also have strategies and tools in place to detect any hallucinations and inaccurate outputs that manage to slip through.

Leading AI governance platforms and foundation models now include the capability to detect hallucinations. IBM® watsonx.governance™ and IBM’s latest Granite Guardian release (IBM® Granite™ Guardian 3.1, part of IBM’s family of Granite language models designed for businesses), both evaluate generative AI models’ performance on metrics such as answer relevance and “faithfulness.”

“They call it ‘faithfulness,’ which is the opposite of hallucination,” Varshney explains. “When a response is faithful to its source documents, it is not hallucinating.” Granite Guardian 3.1 also features capabilities for detecting counterproductive AI use and output, such as jailbreaking, profanity and social bias.

AI governance tools notwithstanding, humans still have key roles to play in preventing the spread of AI-generated misinformation. As companies implement AI systems, they should consider where they’re establishing control points that allow for human oversight, Candy says. “We need to be purposeful in designing points in the process where there’s human interaction, human checks and balances and human decision-making.”

Such human decision-making is especially important, he notes, with the advent of powerful AI agents that can tackle increasingly sophisticated tasks.

For example, while a life sciences company might use multiple AI agents to research and write compliance reports on new drug development, the company would still assign a human employee to review and fact-check the reports before filing them with a government agency.

“Ultimately, you’re still going to have a human in the loop around that process, checking and validating,” Candy says. “I would not underestimate the importance of the human piece.”