The hidden costs of knowledge fragmentation

Imagine an employee racing to answer a customer’s urgent request, only to find three conflicting policies and no clear source of truth. The result? Delays, frustration and risk to the customer relationship. This issue is called knowledge fragmentation: information exists, but it’s spread across systems, versions and people. Employees face uncertainty and rework, while AI systems risk spreading outdated or inconsistent guidance even faster.

Building an AI assistant or agent is easy but ensuring it uses accurate, approved information is much harder. As these tools become common, organizations must verify that answers are based on trusted, up-to-date sources. Therefore, the goal is not simply to make knowledge searchable. It is to give people and AI systems a dependable way to find, verify and use the information behind their decisions.

Gartner reports in 2023 that 47% of digital workers struggled to find the information needed to do their jobs. A separate Harvard Business Review study followed the search patterns of 137 employees across three Fortune 500 companies. The researchers found that workers moved between applications and websites about 1,200 times a day, spending just under four hours each week getting back on track after those switches.

Lost time is only the most visible cost. Different versions of the same policy or guidance can lead to inconsistent decisions. A support representative might give a customer an outdated answer, a seller might build a proposal with old product information or a manager might approve a request without seeing a relevant exception.

The common workaround is to ask the person who knows where everything is. Over time, this turns experienced employees into human help desks and makes answers dependent on their availability. When they leave, much of that context leaves with them. What began as a search problem becomes a coordination problem, with people operating from different assumptions.

Moving every document into one repository sounds tidy, but it is rarely practical. Customer records, policies, technical guidance and project decisions live in different systems for good reasons. The goal should be to make trusted knowledge usable wherever it resides, rather than force everything into one place.

Enterprise search is useful when someone wants to explore a topic, compare documents or review several sources. But when an employee needs to answer a question, make a decision or complete a task, a list of results is often only the start. The harder problem is connecting that employee to an answer they can verify and use.

This is where AI knowledge agents can play a different role. A knowledge agent can draw on enterprise information to answer a question in context, show the sources behind the response and help a user move from finding information to acting on it. Rather than asking employees to interpret several documents themselves, a knowledge agent can bring relevant evidence into the answer while respecting the permissions already attached to the underlying information.

Many AI knowledge agents use retrieval-augmented generation (RAG) to ground responses in enterprise content rather than relying solely on a model’s training data. That aspect can make answers more relevant, but retrieval alone cannot make weak knowledge trustworthy. If the wrong document or an outdated policy is retrieved, the answer can still be wrong. Trust depends on whether the underlying information is current, appropriately governed and suitable for the question being asked.

It’s easy to build a convincing RAG demo with a handful of documents and carefully chosen questions. But real-world enterprise systems must handle thousands of documents, inconsistent permissions, vague questions and important information buried in tables, diagrams, scanned forms, recordings or conflicting versions.

Therefore, retrieval is only part of the challenge. A language model can reason only over the information it receives. This means that the right answer might be out of reach if the wrong source is retrieved or important context is lost during extraction. The information itself is also constantly changing as policies are revised, products evolve and teams reorganize.

Reliable AI knowledge agents depend on more than connecting a model to content. Organizations need accountable owners for important sources, clear permissions, rigorous testing and ongoing monitoring. They need to understand where an answer came from, recognize when the available evidence is not sufficient and correct problems before unreliable information becomes embedded in everyday work.

These questions become more important as different teams begin creating agents with access to different sources, models and actions. Leaders need visibility into which agents exist, who owns them, what information they can access and whether the answers they provide remain accurate over time.

A wrong policy used by one agent is a knowledge problem. However, if several agents begin deciding or triggering workflows based on that same policy, it becomes an operating problem. As AI moves beyond answering questions and starts supporting decisions and workflows, the quality of those outcomes will increasingly depend on the quality of the knowledge behind them.

Most organizations already have much of the information needed to answer common questions, guide decisions and support everyday work. The opportunity is to make that collective knowledge easier to find, verify, govern and apply consistently, so AI knowledge agents can extend expertise without also extending uncertainty.

For organizations exploring AI knowledge agents, a practical place to start is with a question that regularly sends employees across documents, systems or colleagues looking for an answer. A strong first use case has an approved source, a clear owner who can keep that source current and a business outcome that can be measured.

Before introducing an agent, capture what the experience looks like today. Then, test the agent with the questions people ask, including vague requests and situations where the correct response is “I don’t know.” The important measure is not simply whether the agent produces an answer, but whether employees can trace that answer to an approved source, complete the intended task and avoid an unnecessary escalation.

Review the misses with the people who own the underlying information, improve both the source and the agent, then repeat the process for the next question. Scale comes from that repeatability: a common question, an approved source, a clear owner and an outcome that can be measured.

That repeatability is how an AI knowledge agent becomes more than a convincing demo. It becomes a dependable way to help people resolve real work with information they can understand, verify and trust.