AI learns to see hidden cancer signs

Scientists have trained an AI to detect invisible immune activity within tumors using only standard pathology slides.

Researchers at Microsoft Research, in collaboration with universities, recently developed GigaTIME, an AI system that examines ordinary pathology slides to infer the invisible internal states of immune cells within tumors. It is part of a broader effort, including work at IBM Research, to build AI that models biology as a spatial system rather than a collection of isolated data points.

“What we are trying to do is use very readily available data to infer information that is otherwise hard to get,” Hoifung Poon, General Manager at Microsoft Research, told IBM Think in an interview. “That opens the door to studying immune behavior at a scale that was not practical before.”

The technical challenge is substantial. Standard pathology slides are stained with hematoxylin and eosin, a technique that reveals tissue structure and cell shape. Under a microscope, a pathologist can identify different cell types by their appearance. But they can’t determine whether an immune cell is activated, exhausted or suppressed, even though those states help determine whether the immune system is fighting the cancer or failing to do so.

“If you put tumor tissue under the microscope, you can roughly make out different cells based on shape,” Poon said. “The difficulty is that you don’t know the internal state of the cell.”

Obtaining that information through conventional means requires specialized techniques such as multiplex immunofluorescence, which can take days to complete and cost USD thousands per sample. A standard pathology slide costs a few dollars.

GigaTIME was trained on tissue samples that had been analyzed both ways: standard slides and expensive immune measurements. The model learned which visual patterns correspond to which immune states, and can now infer one from the other.

The system’s design reflects a growing recognition in AI research, from medicine to materials science, that spatial relationships carry information that disappears when data is averaged away. Averaging immune cell counts across an entire tissue sample, for instance, erases the local interactions between cells that help determine how the immune system behaves, Poon said.

“The spatial arrangement tells you something that you cannot see if you just average everything together,” Poon said. “Where the immune cells are, who they are interacting with and how they are organized really matters.”

Poon described the spatial patterns as a kind of biological grammar. “The patterns tell you something about the status of the cancer and what the immune system thinks about it,” he said.

At IBM Research, teams have been pursuing a parallel approach, developing multimodal AI systems that integrate imaging, molecular data and clinical information. The goal is to capture biology, not as isolated measurements, but as an interconnected system that operates across multiple scales.

“Biology is not one layer of information,” Ajay Royyuru, Chief Science Officer for Healthcare & Life Sciences Research at IBM, told IBM Think in an interview. “It is multiscale, it is organized and it is contextual. If you only look at one slice, you miss what is actually driving behavior.”

Royyuru emphasized that spatial context is fundamental to cellular function in ways that AI systems are only beginning to reveal. “A cell sitting next to another cell can communicate directly,” he said. “That same signal is not available to a cell ten cells away.”

Both researchers stressed that these systems are research tools meant to guide experiments and generate hypotheses, not substitutes for laboratory work.

“This is not about replacing experiments,” Poon said. “It is about using AI to guide where you look and what questions you ask.”

The distinction matters because the biological systems these AI tools analyze remain deeply complex. Cancer, the domain in which GigaTIME operates, can evolve under selective pressure and adapt to treatment in ways that continue to confound researchers.

“You wipe out most of the cancer cells, but if one resistant cell remains, it can come roaring back,” Poon said. “That is why cancer keeps recurring.”

Royyuru suggested that the field is only beginning to understand what AI can contribute to biological research, and that humility remains appropriate. “We are studying a system that has had billions of years to evolve,” he said. “It is not obligated to reveal itself to us just because we can measure more things.”