The essential guide to agentic AI

Introduction

Building for an agentic AI edge

Organizations are ready to embrace the promise of agentic AI, but scaling its impact demands more than applying AI to old processes or tweaking existing systems. Agentic systems are autonomous by design: they act, learn, and adapt across workflows in real time. As discussed in previous IBM Institute for Business Value (IBM IBV) research, agentic AI needs a new operating model—engineered from the ground up for real-time strategic decisions and seamless human-AI collaboration across domains. It's time to build the high-performing machine of the future—where agents operate autonomously in cooperative multiagent systems across enterprise operations.
 

Agentic AI demands a new operating model, engineered from the ground up for real-time, strategic decision-making.   

 
The competitive gap is already forming. The IBM IBV surveyed more than 1,000 global C-suite executives and found that while most organizations are exploring agentic AI, few are ready to scale it. Across every critical element of the operating model—from infrastructure to governance to workforce readiness—fewer than half of enterprises report they are scaling or optimizing today.

The ambition is there: 80% of executives are increasing investments in agentic AI, with spending projected to nearly triple by 2027. But without the right technical frameworks, data strategies, and execution plans, much of that investment will remain trapped in proofs of concept—delivering pockets of value without the enterprise‑wide transformation envisioned by leadership.

Over the next two years, 72% of executives expect agentic AI to enable new technology capabilities that can transform business models and industry structures.

 
Our analysis identifies a group of more advanced organizations that are translating investment into impact in customer engagement and innovation. They combine committed leadership, advanced operating models, and thoughtful human-AI collaboration for purposeful integration of sophisticated agentic systems. They're more confident in multiagent capabilities, more strategic in model management, and significantly ahead in building scalable, interoperable foundations.

Developed in partnership with Amazon Web Services (AWS), this report will help organizations at all maturity levels advance their readiness for autonomous agents. We examine three essential components that determine whether agentic AI becomes a performance engine:

• First, the chassis, or the strategic technical framework. It’s the secure, open framework that connects agents, models, and systems enterprise-wide, supporting orchestration, interoperability, and scalability.
• Second, the fuel, or the high‑quality, governed data streams that power adaptive decision‑making through integration, accessibility, and real-time flow throughout the enterprise.
• Third, the powertrain, or the human-AI collaboration that converts potential into measurable outcomes by aligning actions to KPIs and embedding accountability.

We conclude with an action guide that outlines strategic steps to take now in preparation for agentic AI. Leaders who get these elements right can adapt faster, innovate more deeply, and deliver impact at enterprise scale. The rest will be left optimizing yesterday’s operating model.

Perspective

Part one

The chassis: Why is agentic AI architecture critical for a multimodel world?

Agentic AI changes the rules for enterprise architecture. Traditional AI often relies on preprogrammed algorithms in point solutions—sometimes powered by a single model—solving specific tasks in isolation. Generative AI and its use of large language models requires more robust infrastructure to handle multimodal (text, audio, video) capabilities and larger computational demands. Autonomous agents demand something fundamentally different: dynamic interaction across multiple models and modalities, real‑time information exchange, and contextual task‑switching capabilities. Success requires architectures built for orchestration, where agents coordinate autonomously across platforms and systems without constant human intervention. 

Most leaders (76%) know they need an open, secure-by-design architecture, but few have built it.

 
Most leaders understand this foundation is critical, but construction is still in the early stages. Three in four (76%) agree that an open, secure‑by‑design architecture is essential for multimodel agentic AI environments, where network effects emerge from collaboration and data sharing between diverse models and agents. More than two-thirds (69%) say this type of architecture promotes transparency, interoperability, and faster innovation. Yet when it comes to implementing interoperability and scalability features, fewer than one-third have done so to a great extent. 
 

Few organizations have the interoperability and scalability architecture features needed for agentic AI.
Percentages represent organizations that have each feature to a great extent.

 
The technical foundation gaps between the most and least mature organizations are substantial. 89% of the advanced group ensures seamless integration of diverse AI models, versus 58% of the least mature group. And 85% have scalable infrastructure to support complex AI workloads, versus 52% of their less advanced peers. These significant differences represent the divide between organizations ready to scale and those still struggling with basic model integration.

In a fast‑changing environment where organizations deploy a diverse mix of custom models, third-party foundation models, and open-source alternatives, architecture must deliver these core capabilities:

• Interoperability and orchestration. Agents must share information, coordinate actions, and hand off tasks across different systems and model types. 62% of leaders understand their architecture requires orchestration layers that manage data flow, model interactions, and dynamic task switching, but 61% say seamless integration and communication between models is a major challenge. A model-agnostic architecture, such as container-based orchestration platforms that package models into standardized, portable units, can support multiple model types and help ensure they operate together without friction.
• Scalability and flexibility. Elastic scaling is essential for agentic AI’s diverse workloads. 47% of executives are turning to cloud platforms for scalability and flexibility, while 44% are conducting infrastructure assessments to close capability gaps. Cloud and hybrid architectures let compute‑heavy models run alongside lighter ones, integrate new agents quickly, and connect securely to ecosystem partners—balancing compliance and control with reach and adaptability to accommodate evolving demands.
• Security. Executives report data privacy and security is the number‑one challenge to implementing agentic AI, yet only 38% have embedded security into every stage of the AI lifecycle—from development to deployment. A secure‑by‑design architecture protects sensitive data, enables compliance management, and creates the foundational layer for autonomous agents to operate safely. It includes integrating encryption, access controls, and threat detection directly into the infrastructure, so security scales automatically as agents are deployed across the enterprise. 
• Trust and transparency. Autonomous decision‑making demands visibility and accountability. 73% of executives expect AI agents to enable accountability and transparency in decision‑making by 2027. They will need technical architectures that can monitor agent behavior in real time, log decision pathways, and provide audit trails that explain how and why autonomous actions were taken. Without built‑in transparency, even the most sophisticated agents become "black boxes" that can erode trust and limit adoption.

In the agentic AI era, your architecture will define how quickly you scale, how effectively you innovate, and how well you compete. Winners make intentional architecture decisions—building the secure, open, and adaptable framework that turns agentic AI pilots into enterprise‑wide value.

“Building ethics analysis into agentic AI isn’t just about doing the right thing—it’s about preventing these autonomous systems from making decisions that conflict with human values at machine speed and scale.”

Chief Technology Officer, Electronic products manufacturer

Perspective

Part two 

The fuel: How to enable a continuous flow of high-quality data

Data is no longer just input. With agentic AI, its role shifts from learning patterns and feeding predictions to becoming the continuous fuel stream that powers autonomous, goal-driven action in dynamic environments. Agents need data flows that update in real time as agents and people execute workflows, creating feedback loops that improve both decision‑making and operational outcomes. This constant cycle enables agents to adapt dynamically and deliver context‑rich insights that drive sustained business impact.

The stakes of data management are higher with agentic AI. In autonomous systems, data functions as a stream of real-time experience, so its quality, timeliness, and governance are exponentially more critical to performance than for traditional or generative AI. For example:                  

• Inaccurate or incomplete data can mean an agent takes an unpredictable action, such as an inventory management agent automatically over-ordering massive quantities of materials based on faulty demand forecasts. 
• Stale data can mean an agent acts on outdated context, such as a trading agent executing a large buy order based on a stock price that is two minutes old, missing a sudden price crash. 
• And lack of advanced governance can lead to unauthorized, irrevocable actions taken in the real world, such as an irreversible series of trades that cause significant financial loss.

Organizations have built strong data foundations—privacy, governance, accessibility, and integration are priorities for most. The challenge now is evolving those foundations for agentic AI: making data faster, more connected, and ready for agents that learn and adapt in real time. While 60% say their practices are future‑ready, they need to turn readiness into a competitive advantage by embedding agentic‑specific requirements into their data strategy now.
 

Organizations have solid data management practices.
Percentages reflect executive agreement with each data management statement.

Organizations with advanced maturity have embedded these principles more deeply, such as seamless data integration (73% versus 64% of the least mature). More tellingly, 84% say effective AI governance is essential for agentic AI success (versus 65%). 

Enabling the flow of data for agentic AI depends on three strategic priorities:

• Data governance that enables agility. In the agentic AI era, governance must enable speed. That means embedding rules and controls directly into data workflows so the right agents can access the right data instantly, without manual bottlenecks. Automated data governance can enforce quality rules without delays, provide role‑specific access and permissions to meet each agent’s needs, and maintain automated lineage and cataloging so agents can evaluate and understand data reliability.
• Integration across silos. Structured and unstructured data trapped in silos loses value. Seamless integration requires two key capabilities: automated data preparation that standardizes formats and ensures quality as information flows between systems, and unified access layers that let agents combine transactional records, operational metrics, partner data, documents, and multimedia content into holistic views that support better decisions and personalized experiences. Modern data integration tools such as a data fabric or data lakehouse—a combination of a data lake and data warehouse—help dissolve barriers with automated data preparation and unified access layers. Yet in separate IBM IBV research, only 19% of COOs report their organizations have fully developed the components of an enterprise-wide data architecture and scaled data integration across functional areas.
• Accessibility for continuous learning. Once data is unified, it must support constant agent improvement. 77% of organizations expect their AI agents to continuously improve performance over the next two years through learning, but this requires speed, security, and feedback. True accessibility—and adaptability—means real‑time data availability, embedded governance in access controls, and closed‑loop feedback systems where agent outcomes feed directly back into training datasets. These capabilities enable agents to adapt to changing conditions and deliver ever‑better results. 

 
A high‑performing fuel line is built for velocity, reliability, and adaptability. Organizations that master continuous, well‑governed data flows will give their agents the power to act with precision, learn faster, and deliver outcomes that compound over time. Without a robust fuel line, agentic AI will sputter.

“How can we speed up decision-making? We’re in the fast-moving consumer goods space where an average product cycle from arrival in our distribution center to shipping is 8–9 hours. It would be a huge step ahead if we could use agentic AI to do logistics optimization.”

Chief Supply Chain Officer, Consumer goods company

Case study

Part three

The powertrain: How people and agents partner for impact

Execution in the agentic AI era demands more than new roles and skills. It requires redefining what success looks like for the business, people, and agents. Nearly half of executives (45%) say how they set business goals and objectives will need to change because of agentic AI. Why? Because autonomous agents don’t just accelerate existing processes—they also open the door to new opportunities that weren’t possible before. Over the next two years, 72% of leaders expect agentic AI to enable new technology capabilities that will transform business models and industry structures.

But goals alone won’t deliver impact. The real powertrain is human-AI collaboration, a new approach that blends autonomy with accountability to convert potential into performance. Unlike traditional AI, which often positions the workforce as supervisors of static systems, agentic AI thrives when people and agents collaborate: people provide judgment, creativity, and oversight, ensuring decisions align with strategy and values, while agents surface insights, execute actions at speed and scale, and adapt in real time. The partnership is adaptive, shifting gears based on conditions, priorities, and performance data.

Executives value human-AI collaboration. 70% agree people and agents learn from one another, and 64% agree people and multiagent systems work in tandem to drive strategic objectives.

Yet readiness to execute at scale remains limited. Only 42% of organizations are scaling or optimizing workforce requirements for AI‑related roles, and just 37% are doing the same for performance metrics and impact evaluation. Less than half have taken steps to measure agentic AI’s impact on their organization. Without these foundations, collaboration may be high on intent but low on quantifiable impact.
 

By 2027, less than half of organizations expect to have taken steps to measure the impact of agentic AI.
Percentages represent those executives saying they will have taken each step to maximize agentic AI's impact by 2027.

High‑performing human-AI collaboration depends on these mandates:

• Collaboration aligned to outcomes. Human-AI collaboration must be tied directly to KPIs and business goals. When both people and agents are accountable for shared results, every decision and action is focused on delivering value. Almost half (46%) of organizations have developed KPIs to monitor the impact of AI agents on business performance.
• Embedded autonomous performance measurement. Real‑time monitoring of workflows, experiences, and outcomes helps ensure that impact is visible and decisions can be adjusted quickly. This requires systems that track agent performance alongside people’s contributions. 45% of organizations have assigned performance metrics to individual AI agents.
• Trust and ethical alignment. 57% of organizations expect their personnel to work within their AI governance structure to manage ethical challenges. But their frameworks for managing ethical AI practices are the least mature aspect of their operating models: only 30% are scaling or optimizing them. And lack of a formal AI governance framework is their number-two barrier to implementing agentic AI. Trust requires clear role definitions in which people set ethical boundaries and monitor agent behavior, while agents operate within those parameters and flag ethical dilemmas for human review. Organizations need to prioritize embedding governance controls that prevent agents from exceeding their authority and training their talent to recognize and respond to ethical cases.
• Workforce readiness for orchestration. Preparing the workforce to guide, refine, and learn from autonomous agents is essential. Executives understand this, with 71% saying they are constantly fine-tuning the relationship between people and agents. 70% say people are always in the loop on high-complexity, high-impact decisions. And as agents continually evolve and take on new responsibilities, change management must be ongoing, not a one-time transition. Roles, processes, and governance must adapt in lockstep with agent capabilities, ensuring accountability, oversight, and collaboration remain aligned with business objectives.

 
The most mature organizations are more deliberate in managing the human-AI relationship. For example, 77% ensure they protect and value employees’ role in critical thinking (versus 66%). This intentional approach strengthens adoption and trust, but even the advanced group has been slow to define KPIs that measure the impact of agents, or the impact of agentic AI on their employees, customers, and business partners. Organizations that invest in outcome-aligned collaboration and real-time performance measurement—along with a skilled pit crew—can turn agentic AI from a promising technology into a competitive advantage that compounds over time.

“My focus is on creating an agile organizational structure. Roles are changing, not just for teams—stakeholders too need to realize they will change. Everyone’s looking to the workforce to embrace AI but, when they do, that changes the foundation of the enterprise as well.”

Chief Operating Officer, Medical devices and disease diagnostics

Perspective

Conclusion

Early movers set the pace

Organizations are pouring resources into building agentic AI solutions, in search of greater productivity, operational efficiency, and customer satisfaction—and ultimately, revenues and profits. Our analysis shows that organizations further along the operational maturity curve already report stronger innovation and customer engagement than their less mature peers, and they are more ready for future impact. Three in four expect innovation with agentic AI to enable new capabilities that will transform their business models, versus 64% of the least mature group.

Frontrunners are cementing their ability to scale agentic AI with strategic decisions—knowing when to push, when to conserve, and when to pit. We also see that their leadership plays a decisive role. In the highest‑maturity group, 80% say leadership prioritizes AI as a key strategic component (versus 64% of the least mature), 78% report active C‑suite involvement in setting AI vision and strategy, and 78% have clear goals and metrics for AI success. 

These advanced organizations are positioned to adapt faster, innovate meaningfully, and compete intensely—lap after lap—in the agentic AI era. The rest must shift from a focus on fine-tuning to rebuilding their AI engines with renewed intention and speed.

Action guide

Turning vision into measurable progress

Building maturity in your agentic AI operating model doesn’t require sweeping change overnight—but it does demand focused, intentional steps that strengthen your architecture, data strategy, and human-AI collaboration. The following actions can help you move forward in the near term.
 

1. Build the architecture for agentic AI

Start by assessing your current architecture for scalability, interoperability, and security gaps.

• Leverage existing cloud or hybrid infrastructure for agentic AI by mapping where autonomous agents will run, what data they’ll need, and how they’ll connect across environments. Extend and integrate existing capabilities—for example, API gateways or container orchestration—with agent‑specific communication frameworks and messaging layers to achieve scalability. 
• Run a small‑scale orchestration pilot in one workflow or domain to test tools that manage task allocation, data exchange, and performance tracking between diverse agents and models. Learn and scale across domains.
• Begin embedding security and compliance into agentic workflows by extending current enterprise security frameworks. Define agent identity/authentication, role‑based access to data, and automated logging for audit readiness.
   

2. Enable continuous data flow to and among agents

Conduct a quick audit of your data governance framework to identify bottlenecks that slow access to high‑quality data. 

• Pinpoint high‑impact data sources most critical to agent decision‑making in priority use cases, and integrate them into a unified environment.
• Enable real‑time or near‑real‑time data access for one pilot workflow using streaming or API‑based integration to ensure agents act on the latest conditions.
• Set up a closed‑loop feedback process so the outcomes of agent actions are automatically captured and used to retrain or fine‑tune models, supporting continuous learning.
   

3. Redefine impact of employees and agents

Define or refresh your KPIs to measure outcomes from human-AI collaboration. 

• Redefine KPIs for one or two pilot workflows to measure both workforce and agent contributions. Include metrics for accuracy, decision speed, and adaptability.
• Establish a collaboration framework that defines how people and agents coordinate—who makes which decisions, how hand‑offs occur, and how oversight is maintained.
• Progress autonomous performance monitoring using existing analytics or monitoring tools, and then use workflow impact feedback to track agent activity in real time and flag anomalies.