As organizations increasingly recognize the power and game-changing potential of AI-enabled applications, the mainframe continues to be an essential part of the technologies of tomorrow.

The mainframe's unique strengths—including the ability to process high volumes of transactions, support complex and highly regulated workloads, and ensure security and reliability at scale—make it an indispensable contributor to any AI strategy. By embracing and unlocking the mainframe's potential as a source of reliable, secure, and relevant data and information, organizations can unlock new growth opportunities and maintain a competitive edge in the rapidly evolving digital landscape.

Through the partnership between IBM and Microsoft, organizations can unlock the potential of the mainframe, making its business functions and data available to generative AI-enabled applications running in Azure. Through data services such as IBM® z/OS® Connect and IBM Z® Digital Integration Hub, developers can use build APIs to access data and services on the mainframe. This supports the near real-time exchange of information between core business applications running on the mainframe and digital front-end applications and services running on Microsoft® Azure.

Several different approaches can be used to successfully integrate mainframe data with Azure AI-enabled applications. You can expose mainframe data through APIs, which allows Azure applications to access and manipulate data programmatically, and you can use message queues to integrate mainframe data with Azure applications. You can also use change-data-capture tools to synchronize mainframe data into Azure data stores. You can use cloud-based integration platforms such as Microsoft Azure Integration Services to integrate mainframe data with Azure applications.

When it comes to integrating mainframe data with Azure applications and unlocking the full power of AI for competitive advantage, choosing the right approach is critical. Each option has strengths and weaknesses. The right choice for you will depend on your organization’s specific needs and requirements. By carefully evaluating the benefits and limitations of each approach, IBM and Microsoft can help you select the best solution for your digital transformation journey.

We will explore these solutions through a fictional US bank that has embarked on a journey to modernize its mainframe applications. The bank intends to take full advantage of the power of AI and generative AI (gen AI) to build a whole new experience for its customers. By looking at how this fictional bank approaches the problem, you’ll gain insights into some of the strategies and approaches that can help your organization achieve success.

To enhance customer service, our example bank wants to create an Azure-based application that uses Azure AI for advanced analytics and personalized financial advice and presents the results back in rich text format (RTF). The application needs to integrate up-to-date proprietary information kept on the mainframe with public market and securities data.

The application will provide customers with investment recommendations and insights unique to the bank’s investment approach. The bank wants to keep its proprietary information on the mainframe and avoid the cost and complexity of migrating the application and large volumes of data to the cloud.

As the lead AI developer, Sally is aware of retrieval-augmented generation (RAG), which enhances the capabilities of large language models (LLMs) by integrating real-time information retrieval into the text generation process. This approach allows AI models to fetch relevant data from external knowledge sources, ensuring that the generated responses are accurate, up-to-date and contextually relevant. While discussing this approach with her leadership team, Sally highlights its key benefits: while the mainframe data remains on the mainframe, with no change to existing systems and applications, the data can be accessed and by an AI-powered application on Azure. For more details, see here.

Retrieval-augmented generation (RAG) is an architectural pattern that enables foundation models to produce factually correct outputs for specialized or proprietary topics that were not part of the model's training data. By augmenting users' questions and prompts with relevant data retrieved from external data sources, RAG gives the model “new” (to the model) facts and details on which to base its response.

The RAG approach ensures compliance with stringent regulatory requirements and enhances data security, allowing the bank to deliver precise, contextually relevant responses derived from the most current mainframe data while simultaneously mitigating the risks and costs associated with data migration.

As the application starts to gain traction in the market, the bank sees an opportunity to add new capabilities and features to remain ahead of the competition. Sally and her development team become increasingly concerned about the complexity of the parsing and indexing techniques that are required to fetch the needed data in real time, especially given the complex, multi-part queries that synthesize the information and present it back to the calling application.

Sally and her team discuss how they could improve the model given its complexity and realize a new approach is needed to scale the solution. One of her teammates, James, outlines an alternative— an agentic-RAG approach where “agents” take responsibility for collecting, processing and interpreting data and then return the required information to the model or calling agent.

James outlines how an agentic-RAG based approach to mainframe cloud integration offers several benefits over the current RAG solution. By using autonomous software agents running on the mainframe, an agentic-based approach enables an agent on the mainframe to execute complex, multi-step query actions. In this case, the agent would combine the bank’s proprietary forecasting model with customer-specific data, iterating over the model before returning the requested data to the Azure-based application. Agentic-based solutions provide greater flexibility and adaptability, which allows the bank to easily extend and add functionality—just by adding a new agent. James also outlines how an agentic-based approach would provide better security and compliance features, ensuring that sensitive information is protected and regulated data is handled in accordance with relevant standards, see here. 

An agentic AI platform consists of an LLM that orchestrates the behavior of multiple agents that can be deployed across various applications. These agents might be more AI models or simple search tools that can quickly look up information in a knowledge base or online.

Designing a set of autonomous agents to retrieve financial data, execute data processing tasks and provide a summary back to a calling Azure application is a complex task. Further extending this analysis with agents in Azure (which augment the results) adds to that complexity. It requires careful planning, coordination, design and implementation. But this approach provides the basis for a robust and scalable system that can provide accurate and timely summaries of financial data, risk management and investment analysis at scale while using data sources in Azure, on the mainframe and elsewhere.

As the solution develops, the bank seeks to replicate some of its mainframe-based data to Microsoft Azure to improve data analytics and decision-making capabilities. The bank also wants to further integrate the information with data sourced from a wide range of applications and services running on Azure.

The bank requires a real-time data replication solution that ensures high availability, scalability and compliance with industry regulations. In partnership with Microsoft and IBM, Sally and her team turn to Microsoft Fabric—a comprehensive analytics solution that is designed to integrate various data sources into a unified platform.

For organizations that rely on mainframes, such as those that use IBM® Db2® for z/OS®, Microsoft Fabric offers tools to seamlessly bridge the gap between enterprise systems and modern cloud-based applications. By using components such as dataflows, data pipelines, and data gateways, Fabric helps Sally and her team secure real-time data access and movement from mainframes to Azure.

Replicating mainframe data to Azure can be an appropriate solution when that information is to be combined with other data not residing on the mainframe. In these cases, a secure, low-latency and seamless approach that combines Microsoft Fabric with IBM’s rich toolset provides an optimal approach to this hybrid solution. See here for more information.

In today's fast-paced and increasingly complex banking environment, financial institutions are under pressure to provide seamless customer experiences, improve operational efficiency and reduce costs—our example bank is no different. The bank sees AI at the edge—where AI models are deployed on devices or systems closest to the source of the data—as a way to reduce latency and improve real-time decision-making. The bank's branches are an ideal example of where AI at the edge can make a significant impact; branches are the frontline of customer service, where customers interact with bank representatives, deposit and withdraw cash, and conduct various financial transactions.

With the increasing use of digital channels and the integration of AI into so many devices, branches are evolving from mere transaction points to advisory centers where customers seek personalized advice, guidance and intelligence. AI at the edge can make all the difference. For more information, see here. 

Edge AI refers to the deployment of AI algorithms and AI models directly on local edge devices such as sensors or Internet of Things (IoT) devices, which enables real-time data processing and analysis without constant reliance on cloud infrastructure.

Sally and her team are aware of Microsoft and IBM’s partnership around cloud-based AI, edge AI, mainframe-based AI, and support for IOT devices. They are also aware of where and how this convergence and the aggregation and analysis of data—through a combination of agentic AI, RAG and data replication/data fabric strategies—could help the bank unlock real-time insights, improve efficiency and drive innovation, delivering real value and a differentiated, personalized experience to customers.

IBM’s recent announcement reflects a growing need to support more advanced AI models on the horizon. AI models are evolving to include gen AI and LLMs into a blended approach called “ensemble AI.” IBM’s investment to support this evolution indicates that there may be opportunities to bring ensemble AI to the mainframe at scale. This would open a new generation of AI use cases on the mainframe and expand the opportunities and potential for the bank to bring value to customers through mainframe modernization on Azure.

Mainframes continue to be repositories for much of organizations’ most sensitive and critical data. As companies look to modernize, adopting a fit-for-purpose strategy that places workloads on the most appropriate environment—whether that be mainframe, public cloud or edge—allows them to leverage the relative strength of these platforms.

IBM has significantly invested in integrating AI capabilities into mainframe systems, recognizing the potential to enhance transactional workloads, bolster cybersecurity and streamline operations. By embedding AI directly into mainframe applications, IBM enables businesses to extract valuable insights and drive innovation. Not only does this maximize the value of existing mainframe infrastructure, but it also ensures that critical workloads are processed with unmatched security and reliability. The integration of AI helps modernize mainframe operations, making them more efficient.

Complementing these advancements, IBM’s collaboration with Microsoft Azure provides a robust framework for mainframe application modernization, leveraging Azure’s scalable cloud infrastructure and analytics alongside the mainframe’s reliability and security. This partnership enables businesses to seamlessly transition to a hybrid cloud model, combining IBM’s AI-driven solutions with Azure’s services to enhance agility, reduce operational costs and accelerate digital transformation.

Learn more by reading the Mainframe Modernization and AI study by IBM Institute for Business Value, (IBV) in collaboration with Oxford Economics.