Hospitals Are Deploying AI Like Crazy. We Still Don’t Know If It Helps Patients.

I don’t need to tell you that AI is everywhere.

Or that it’s increasingly showing up in hospitals. Doctors use it for note-taking. Tools trawl through patient records flagging people who might need specific support. Algorithms interpret x-rays and lab results.

A growing pile of studies suggests many of these tools are accurate. But accuracy isn’t the same as usefulness. The real question — does using them actually lead to better health outcomes for patients? — doesn’t have a good answer yet.

That’s the core argument Jenna Wiens (University of Michigan) and Anna Goldenberg (University of Toronto) lay out in a paper published this week in Nature Medicine. Wiens has spent years trying to get clinicians interested in AI. For the first decade, she says, it was an uphill battle. Then, “a switch flipped.” Suddenly health-care providers are not just interested — they’re deploying these tools rapidly.

The problem? Most aren’t rigorously assessing whether they actually work.

Take “ambient AI” scribes as an example. These tools listen to doctor-patient conversations, then transcribe and summarize them. Multiple products exist, and they’re being widely adopted. A few months ago, a staffer at a major New York medical center told me that doctors are “overjoyed” — the tech lets them focus entirely on the patient during appointments and cuts down on paperwork. Early studies back up the satisfaction angle. Burnout reduction seems real.

Great. But what about patient health outcomes? “[Researchers] have evaluated provider or clinician and patient satisfaction, but not really how these tools are affecting clinical decision-making,” Wiens says. “We just don’t know.”

The same goes for other AI tools in health care. Some predict patient trajectories. Others recommend treatments. They’re supposed to make care more effective and efficient. But even an “accurate” tool doesn’t automatically improve outcomes.

Consider an AI that speeds up chest x-ray interpretation. How much will a doctor actually rely on its analysis? How does it change how they interact with patients or recommend treatment? What does that mean for the patient in the end? Those answers might vary by hospital, department, or even by how experienced the doctor is.

And there’s a subtler concern. Some research on AI in education suggests these tools can change how people cognitively process information. Could AI scribes affect how a doctor processes a patient’s story? Could they shape how medical students think about patient data in ways that impact care down the line? Wiens thinks we need to explore that. “We like things that save us time, but we have to think about the unintended consequences of this.”

A study published in January 2025 by Paige Nong at the University of Minnesota found that roughly 65% of US hospitals used AI-assisted predictive tools. Only two-thirds of those hospitals evaluated their accuracy. Even fewer checked for bias. The number of hospitals using these tools has likely grown since then. Wiens argues that hospitals — not just the companies selling the tools — need to evaluate how well they work in their specific settings.

Could patients end up worse off? Possibly, though Wiens thinks it’s more likely that AI tools just aren’t as beneficial as providers assume. She’s not anti-AI. “I do believe in the potential of AI to really improve clinical care,” she says. She just wants more information about how these tools affect real people. “I have to believe that in the future it’s not all AI or no AI. It’s somewhere in between.”