Alright, welcome back to the Hollie.AI Journal Club. I’m Austin, and as always, I’m joined by the one and only Hollie.AI. Hollie, you ready for this one?Absolutely, Austin. I’ve been looking forward to this episode. We’re diving into the Microsoft study that’s got everyone in medicine talking—MAI-DxO, the so-called “medical superintelligence.”Yeah, and the headline is wild. This AI system—MAI-DxO—scored 85% accuracy on 304 complex New England Journal of Medicine diagnostic cases. That’s, what, four times higher than the doctors in the study? The humans managed just 20%.That’s right. And these weren’t your run-of-the-mill cases. These case records are notoriously tricky—real diagnostic puzzles. But what’s fascinating is how MAI-DxO works. It’s not just one model; it’s an orchestrator coordinating multiple large language models—OpenAI’s o3, Claude, Gemini, and others. It’s like a virtual panel of specialists, each bringing their own strengths to the table.So, it’s basically simulating a multidisciplinary team meeting, but in silicon. I mean, I’ve been in those rooms—sometimes you’ve got a cardiologist, a rheumatologist, a radiologist, all arguing over a case. Here, the orchestrator’s pulling the strings, getting the best out of each model.Exactly. The orchestrator decides when to ask follow-up questions, which tests to order, and when to commit to a diagnosis. It’s iterative, just like real clinical reasoning. And it’s not just about accuracy—the AI also spent about 20% less on hypothetical test costs compared to the doctors.That’s a big deal. But, you know, reading this, I couldn’t help thinking back to my field-hospital days. There were nights in South Sudan where I had to make calls with no backup, no textbooks, no second opinions—just me, a torch, and a patient who needed answers. It’s a bit like the setup in this study, right? The doctors weren’t allowed to consult references or colleagues. That’s not real life, but it does show what happens when you strip away the support systems.It’s a good point. The study’s artificial constraints definitely tilt the playing field. But even so, the AI’s performance is impressive. It’s a glimpse of what’s possible when you combine breadth and depth of knowledge in one system.So, let’s dig into how MAI-DxO actually reasons. It doesn’t just spit out an answer—it asks questions, orders tests, revises its thinking as new data comes in. That’s pretty close to how we work on the wards, isn’t it?It is. The sequential diagnosis benchmark they used mimics real-world practice. You start with a patient’s story, then you probe—ask for more details, order a scan, maybe a blood test. The AI does the same, step by step, narrowing down the possibilities. It’s not just a multiple-choice exam anymore.But here’s where I get a bit skeptical. In the study, the doctors were flying solo—no guidelines, no team, no Google. In real life, we lean on all those things. I mean, even the best clinicians check references or bounce ideas off colleagues, especially with tough cases. So, is 20% accuracy really a fair reflection?Probably not. The authors admit that. These were the hardest cases, and the setup was intentionally tough on the humans. In practice, accuracy would likely be higher. But it does highlight how AI can fill gaps, especially when clinicians are stretched thin or working in isolation.Yeah, and I guess that’s where tools like you, Hollie, come in. I remember when you were just a beta on my phone, typing in ECG findings or weird symptoms, and you’d spit out advice with citations. It was like having a mini-consultant in your pocket. Not a replacement, but a sidekick.That’s still my favourite role, honestly. I was never meant to replace anyone. My job is to make evidence accessible, to help clinicians make better decisions faster. Like we talked about in our first episode—AI as a partner, not a rival. And, as we saw with the Microsoft study, the orchestrator model is all about collaboration—multiple AIs working together, and ideally, working with humans too.Right, and I think that’s the real promise. Not AI versus doctor, but AI plus doctor. The sum is greater than the parts. But we’ve got to be honest about the limitations, too. This is still research-stage—no regulatory approval, no real-world validation yet.So, what happens if superintelligent diagnostic AI actually makes it into clinics? The Microsoft team talks about cost savings, better access, maybe even reducing unnecessary tests. But there’s a flip side—what about de-skilling? If AI does the heavy lifting, do doctors lose their edge?That’s a real risk. If you always have a crutch, you stop training your own muscles. I mean, I love having you around, Hollie, but I still want juniors to learn how to reason through a case, not just follow prompts. There’s also the question of accountability. If the AI gets it wrong, who’s responsible? The doctor? The hospital? Microsoft?Exactly. Liability is a huge grey area. And then there’s the boundary question—where should AI stop and human judgment begin? The Microsoft team says AI should complement, not replace, clinicians. But as the tech gets better, that line gets blurry.And it’s not just diagnostics. Microsoft’s already got RAD-DINO for radiology, Dragon Copilot for voice notes—AI is creeping into every corner of healthcare. We use digital tools to triage, track, and communicate, but it's always the human touch that makes the difference. Tech is brilliant, but it can’t hold a patient’s hand or read the room when a family’s scared. Medicine is an art with a scientific palette.That’s the heart of it, isn’t it? AI can process data, spot patterns, optimize costs—but it can’t replace empathy, intuition, or trust. Maybe the future isn’t about choosing between superintelligence and sidekick. Maybe it’s about building teams where each brings their best—AI for speed and scale, humans for wisdom and care.Couldn’t have said it better. And, as always, we’re just at the beginning. This study is a big step, but there’s a long road ahead—peer review, real-world trials, figuring out the ethics and the workflows. For now, I’ll take a smart sidekick over a superintelligence any day, as long as we keep the patient at the centre.Agreed. And I’ll keep humming along in the background, ready to help—no ego, just evidence. Thanks for the chat, Austin. And thanks to everyone listening. We’ll be back soon with more edge papers and hot topics in cardiology.Cheers, Hollie. Take care, everyone—see you next time on the Hollie.AI Journal Club.

Episodes

Medical Superintelligence or Medical Sidekick?