The future is programmable: How generative computing could reinvent software

The idea wasn’t born in a flash. It emerged slowly, across late-night Slack threads and hallway conversations, a quiet rethinking of how machines reason. Somewhere between the chaos of prompting and the aspirations of automation, a new concept took shape. It might redefine not only artificial intelligence but software itself.

The premise is bold: What if we stopped treating large language models like mysterious chatbots and started treating them like programmable infrastructure? IBM refers to this emerging discipline as generative computing—a term and framework developed by its researchers to define a new approach to working with AI models. It’s about reengineering how AI models are integrated into systems, not as unpredictable oracles but as controlled, modular software components. If it succeeds, it could mark a turning point for AI development, software design and enterprise technology.

David Cox, Director at IBM Research, told IBM Think in an interview that he coined the term generative computing to describe the shift he sees taking place in AI development. It is neither a brand nor a product. It is a shift, a movement to treat large language models not as clever chat partners but as programmable elements. Forget the magic tricks. This is software engineering.

“It’s not that LLMs are replacing programming,” he said. “It’s that they are becoming a new kind of programming primitive.”

Today, interacting with a large language model often feels like summoning a capricious oracle. Slightly alter a sentence in a prompt, and the output veers off course. Write an essay-length prompt and hope, pray, cajole. It is artful in the way astrology is artful, elusive, interpretive and occasionally profound. But for banks, hospitals and governments, mysticism does not scale.

“You type something, and you get a different answer depending on how you phrased it,” said Ruchir Puri, IBM Research’s Chief Scientist, in an interview with IBM Think. “It’s like the early days of search. We’re still in the era where a comma can change the output. You can’t run an enterprise that way.”

Puri describes a world in which enterprises struggle not only with hallucinations but with the lack of reliability in how models handle edge cases. “We talk a lot about hallucinations,” he said, “but the deeper issue is that the models aren’t guaranteed to follow instructions. You change a word in a prompt, and you don’t know what you’ll get.” That, he argued, is the antithesis of engineering.

To be clear, no one is dismissing the power of modern models. What is broken, Cox said, is the way we use them. “Prompt engineering isn’t engineering. It’s fiddling. We need a system where we don’t have to hope the model does what we meant and we can program it to do what we mean.”

The premise behind generative computing is simple: treat the model as a function. Instead of burying instructions in verbose essays, developers use a runtime—an orchestration layer that splits prompts into atomic parts, routes them, checks conditions and rewrites failures. Logic is not just implied; it is enforced. Control becomes explicit. Structure returns.

“Think about the internet,” said Cox. “You don’t send raw data down a wire and hope it shows up. You have protocols, retries and routing. That’s what we’re adding to AI.” In practice, this means building layered systems that break down complex tasks into smaller, manageable instructions, each of which is verified before proceeding. “You might have twenty small, focused prompts to the model instead of a single long and complex one,” said Puri. “But now you can log each one. You can retry. You can build fallbacks. That’s what enterprises need.”

That structure also opens the door to testing and validation, two principles that have been long absent from gen AI. “You can write assertions around LLM behavior the same way you do around code,” Cox said. “And if you don’t get the behavior you want, you can ask the model to try again, or route to a different subroutine.”

This idea becomes particularly powerful when applied to safety. Puri says he often hears from CTOs who like the potential of AI agents but balk at their unpredictability. “They’re afraid to let them do anything on their own. What if they hallucinate? What if they send the wrong message or approve the wrong transaction?”

To answer that, generative computing introduces tools like hallucination detection, context validation and compliance-aware processing. “With our runtime,” said Cox, “you can interpose a guardian model, one that checks the output of the main model. If something looks suspicious, it can flag it or ask for another try.”

That kind of layering allows for a level of reproducibility and confidence that today’s prompt engineering can’t provide. Developers can mix traditional code with LLM responses, embedding outputs into larger systems without ceding control.

“It’s not a chatbot,” said Cox. “It’s part of your software stack. You test it like you test any other module.”

This moment, Cox said, compares to earlier epochs in computing. In the 1980s, the introduction of software design patterns, such as Model-View-Controller (MVC), allowed developers to separate logic from interface, creating a modular and reusable foundation for building applications. Generative computing, he believes, represents a similar inflection point.

“We’re going to find patterns,” he said. “Just like MVC became ubiquitous in UI development, we’ll see frameworks for orchestrating LLMs. This is the beginning of a new layer in the software stack.”

That vision of structure underlies much of the generative computing movement. Instead of trying to understand every neuron in a large language model, developers build guardrails that align with enterprise constraints. “We create accountability,” said Puri.

Transparency, Cox said, doesn’t have to mean simplicity. “Your car’s engine is complicated,” he said. “But it’s built inside a safety envelope. When something breaks, you have procedures. That’s what we want for AI. Not mystery. Engineering.”

In technical terms, that means exposing the intermediate steps of a model’s decision-making. The runtime used in generative computing can generate logs, attach metadata and perform validation at every step.

“It’s explanation as a feature,” said Cox. “Not as an afterthought.”

Already, IBM’s Granite models have been tuned to support this kind of modular orchestration. They are optimized for quick, memory-efficient inference, enabling many small queries in place of one massive prompt. This makes them well-suited to a runtime-driven approach.

“You can think of them as building blocks,” said Puri. “Instead of trying to do everything in one go, we call them many times for specific sub-tasks. That’s faster, cheaper and more reliable.”

The benefits are not only technical but also organizational. In one pilot project, an enterprise client used generative computing to build a document classification pipeline. In place of relying on one prompt to summarize a legal brief, they broke the task into nine stages: classification, segmentation, extraction, validation, risk assessment, summarization, formatting, review and approval.

“Each stage was isolated and monitored,” said Cox. “If something failed, it could be retried or corrected. You couldn’t do that with a single prompt.”

Puri believes this kind of structure will become the norm. “We’ll stop thinking of LLMs as end-to-end magic and start treating them as infrastructure,” he said. “It’s not about replacing developers. It’s about giving them new tools.”

One of those tools, Cox noted, is the LLM intrinsic, a new concept where special model functions are exposed directly to the runtime, enabling deeper integration and real-time adaptation. “You can attach an adapter that changes how the model behaves,” he said. “That lets you shift tone, reduce risk, even detect hallucinations on the fly.”

These advances could change how software is written. Cox imagines IDEs that include runtime orchestration templates for LLMs, unit tests that validate prompts and version control systems that track model behavior.

“Software engineers will have to learn new skills,” he said. “But the fundamentals are still there: inputs, outputs, correctness, observability. We’re not abandoning software engineering. We’re upgrading it.”

The researchers anticipate that generative computing will extend beyond its current niche use cases. As the field matures, new layers of abstraction, new standards and new job roles will emerge.

He pauses for a moment. “We’ve spent a decade learning how to make these systems sound intelligent,” he said. “Now we have to teach them how to behave.”