AI is Saving Lives by Spotting Cancer Surgeons Cannot See
By Wendy Sutton, Office of the Vice President of Research
Todd Hollon, Research Professor of Neurosurgery
For one in four adult brain cancer patients, residual tumor tissue remains even after surgery. But with the help of artificial intelligence, pioneered by Todd Hollon, Joseph R Novello MD and Alfredo Quinones-Hinojosa MD Research Professor of Neurosurgery at the University of Michigan, that number has dropped to just 4%. Now, surgeons can spot dangerous tissue during surgery that traditional methods often miss, significantly improving patient outcomes.
“We hear a lot about the existential risk of AI, but the best stories around AI don’t get nearly as much press,” Hollon said. “Ultimately, the goal is to help patients. Using this technology, we decrease from 24% residual tumor to 4% using AI. That is a huge drop in residual tumor and relative risk of recurrence. We’re hoping to get that number down to around zero. That’s the next step.”
Gliomas, the most common adult brain cancer, are classified as primary brain tumors, meaning they originate within the brain itself. Unlike tumors that form as a ball-shaped mass with distinct edges, gliomas branch out like fingers, invading into surrounding healthy tissue. This invasive growth makes it difficult to distinguish tumor tissue from normal brain tissue. Ambient white light in operating rooms adds another challenge, making it difficult for even the most experienced surgeons to differentiate the two.
About a decade ago, Daniel A. Orringer, assistant professor of neurosurgery at U-M, and Chris Freudiger, chief technology officer and co-founder of Invenio Imaging, developed Stimulated Raman Histology, or SRH. This technique provides near-real-time microscopic images of brain tissue. Traditional light microscopy required doctors to sample tissue, then stain and section it, embed it in paraffin, place it on a slide and mount it before examining it under a conventional microscope. The process was far too slow for surgery. SRH provided a way to generate images without the need for dyes or labels.
Since then, U-M surgeons have built a library of 4 million images from patients undergoing procedures, collected under Institutional Review Board protocol to ensure ethical collection and use of patient tissue samples. Today, Freudiger leads Invenio Imaging, a startup where SRH imagers are commercially available, and being used not only in brain tumor surgery, but in lung and breast cancer procedures as well.
“We quickly realized that we can take all the great pictures we want,” Hollon said. “But ultimately, we have to do something with them. We have to decide in the operating room about whether we continue to remove tissue or whether it’s normal.”
FastGlioma integrates intraoperative stimulated Raman histology imaging with AI-based analysis to help surgeons assess tumor infiltration during glioma surgery. Fresh tissue sampled from the surgical margin is imaged in the operating room, divided into small regions for analysis, and scored by a vision-transformer model that highlights areas likely to contain tumor. From: Foundation models for fast, label-free detection of glioma infiltration
Surgeons relied on magnetic resonance imaging, or MRI, and fluorescence-guided surgery, which involves injecting a dye into the patient’s vein so that tumor tissue fluoresces. However, intraoperative MRIs require significant infrastructure investment, and neither approach has the sensitivity needed for detecting small fragments of tumor tissue.
To solve this challenge, Hollon led a collaboration with the University of California San Francisco, New York University and the University of Vienna to develop FastGlioma, an AI tool that analyzes SRH images during surgery. Within approximately 10 seconds, it shows surgeons where residual cancer tissue remains and at what density, enabling real-time decisions about how to proceed.
FastGlioma’s foundation model, a type of computer vision AI, was trained on millions of SRH images collected by U-M and collaborating institutions. By processing this vast dataset, the model learned patterns and important descriptive features that distinguish normal brain tissue from tumor tissue. This process is similar to how models like ChatGPT are developed: a general model is trained on large amounts of data to learn underlying structures and is then fine-tuned for a specific task.
In FastGlioma’s case, the fine-tuning step calibrates the model to evaluate surgical images in real time, scoring each field of view in the operating room with a sliding score from zero to one, where zero means no tumor is detected and one indicates dense tumor tissue. A score of 0.25 might signal atypical cells, while scores like 0.5 or 0.75 indicate sparse to increasingly dense tumor tissue. The system also pinpoints the precise location of any residual tumor tissue, giving surgeons a roadmap for their next move. The breadth and diversity of the image library has made the model highly precise at identifying cancer tissue.
When preparing for surgery, Hollon weighs risk-benefit calculations. If the tumor is near an important blood vessel, a critical region of the brain, or a cranial nerve, the surgery cannot safely proceed. Yet it is critical to try to remove as much of the tumor as possible, as leaving it behind will affect a patient’s overall outcome and survival.
Hollon’s career path in AI began during his neurosurgery residency in 2015. With no computing background, he saw a trend toward digitization as hospitals digitized medical records, pathology slides and radiology studies. With the wave of AI improving tools for model training, Hollon dedicated his two-year research residency to machine learning.
“Once I began learning the basic science of machine learning, I fell in love with it,” Hollon said. “I dove in headfirst, had that scientific awe moment and haven’t looked back since. We published papers early on and this huge wave came up all around us. No one could have predicted the success of AI. It permeates our world now and is going to touch every aspect of our lives. It’s really amazing to be a part of.”
Training FastGlioma took about one week using approximately 64 GPUs on NVIDIA 2080 Ti computers. Hollon credits U-M for making the Great Lakes and Armis2 high-performance computing clusters available. Because his data involves patient data, his team requires Armis2, as it is HIPAA compliant.
FastGlioma represents a full vertical stack, a pipeline built on millions of images, a trained foundation model and an edge computing device that delivers results in just 10 seconds, all inside the operating room. Clinical validation is underway, including independent testing at multiple centers to meet regulatory requirements for FDA approval, a key step toward clinical adoption across many types of cancer.
“It’s not just about the ability to detect tumor tissue, it’s about prolonging survival and hopefully preventing recurrence,” Hollon said. “FastGlioma is giving us a preview of a world where AI advances in optics and computations help surgeons make better decisions in the operating room. This is literally a case where computing and AI are saving lives.”
Surgeons at the University of Michigan perform brain surgery, using FastGlioma’s real-time AI analysis of Stimulated Raman Histology (SRH) images to precisely identify and remove cancer tissue.
FastGlioma, an AI-powered tool developed at the University of Michigan, analyzes SRH images to identify and quantify brain cancer tissue during surgery.