University of Arizona astronomer develops novel method to make AI more trustworthy
A new technique developed by U of A Associate Professor of Astronomy Peter Behroozi enables confidence estimates for large neural networks, addressing "wrong but confident" AI outputs across science and society
A University of Arizona astronomer has developed a revolutionary method that could fundamentally change how artificial intelligence models are trained and deployed across science and industry, addressing one of AI's most critical problems: models that confidently deliver incorrect answers.
Peter Behroozi, an associate professor at Steward Observatory, has created a technique that helps AI systems recognize when their predictions might be unreliable – including models with billions to trillions of parameters like those powering today's AI applications. Behroozi's paper, which is awaiting peer review, is available on the open-access arXiv site, the free online archive for scientific preprints.
Development of the technique was supported by a National Science Foundation Early-concept Grants for Exploratory Research grant, which provides funding for high-risk, high-reward research. Now that Behroozi's paper has been posted to arXiv, the accompanying code is available for public use, allowing researchers worldwide to apply the method to their own work.
The method adapts ray tracing – the computer graphics technique used to create realistic lighting in animated films – to explore the complex mathematical spaces where AI models operate.
"Current AI models suffer from wrong-but-confident outputs," Behroozi said. "There are many examples of neural networks 'hallucinating,' or making up nonexistent facts, research papers and books to back up their incorrect conclusions. This leads to real human suffering, including incorrect medical diagnoses, declined rental applications and facial recognition gone wrong."
Behroozi's path to this breakthrough began with his own research in galaxy formation. As creator of the Universe Machine – a computational framework designed to understand how galaxies form by absorbing vast amounts of telescope data – he encountered a persistent challenge: existing methods for exploring uncertainty in complex models simply weren't adequate for the scale and complexity of modern data.
"Galaxies are very complex and so they have a lot of parameters, potentially, that can control what they do," Behroozi explained. "For me, the existing methods weren't working so well to explore how these parameters behaved."
The solution came from an unexpected source: a computational physics homework problem about how light bends through Earth's atmosphere, brought to his office hours by a U of A undergraduate student. That problem – which required simulating how the speed of light changes as it moves through the atmosphere – sparked an idea about ray tracing, the same technique used to create animated films by studios such as Pixar.
"Instead of doing this in three dimensions, I figured out how to make it work for a billion dimensions," Behroozi said.
The new method applies a gold standard technique called Bayesian sampling, which has been used on small models for decades, but has been too computationally expensive to apply to modern neural networks. Rather than relying on a single model's prediction, Bayesian sampling trains thousands of different models on the same data using a special mathematical approach that allows them to explore the diversity of possible responses.
"What's happening is that instead of consulting a single expert, you consult the whole range of experts," Behroozi explained. "If it's something the experts have never seen before, then you'll get a whole range of answers. And you can tell from that that you shouldn't be trusting whatever output is coming out."
Light rays are propagating smoothly through a noisy, high-dimensional space in this artist’s impression. The new ray tracing algorithm improves on previous methods by better averaging information over trajectories, making it many orders of magnitude faster for quantifying uncertainties in large neural networks.
Peter Behroozi
Behroozi's method is many orders of magnitude faster than previous approaches and could lead to safer, more resilient neural networks with substantially fewer hallucinations. The implications extend far beyond astronomy. Individuals and organizations are increasingly using AI in critical decision-making for medicine, finance, housing, energy, criminal justice and autonomous vehicles. Behroozi's method would provide these systems with the ability to recognize when they're uncertain – essentially to know when they don't know.
"Suppose a doctor ordered a routine scan and decided that you needed to begin treatment for cancer immediately, even though you had no other symptoms," Behroozi said. "Many people in this situation would seek a second opinion. The new method would have a similar effect: instead of the opinion of one AI doctor, it would give the range of plausible opinions."
For scientists, the method addresses a pervasive problem undermining trust in AI-assisted research. AI models are being used to design new drugs and materials, predict weather, produce images of black holes, summarize scientific papers and write software – but wrong-but-confident responses remain too common.
"This undermines public trust in scientific output like weather predictions and leads to hesitance among scientists to accept new discoveries that are based on AI models without separate and costly validation," Behroozi wrote in his research summary.
For his own work, the technique opens remarkable new possibilities. Rather than creating simulations that merely match statistical properties of the universe, Behroozi will be able to determine the actual initial conditions of our universe – essentially creating a movie of the real history of cosmic structure formation.
"In the past, what we used to do was just make galaxies in a universe that looks nothing like our own," he explained. "What this technique allows us to do is figure out what were the initial conditions of the actual universe."