Microsoft AI Beats Doctors in Complex Diagnosis, Says Study

Microsoft has unveiled a groundbreaking AI system that it claims can outperform human doctors in diagnosing complex medical conditions. The system, called MAI-DxO (Medical AI Diagnostic Orchestrator), was developed by Microsoft’s AI division under the leadership of Mustafa Suleyman. In a recent study, MAI-DxO demonstrated an 85% success rate in diagnosing highly challenging clinical cases- four times higher than the 20% accuracy rate achieved by experienced physicians under the same conditions.

Unlike earlier medical AI tools that relied on multiple-choice formats like the USMLE, MAI-DxO mimics the real-world diagnostic process. It begins with a patient’s symptoms, asks follow-up questions, selects appropriate tests, and narrows down the diagnosis step by step. This sequential reasoning mirrors how doctors operate in clinical settings, making the AI’s approach more realistic and applicable.

To evaluate its performance, Microsoft used 304 complex case studies from the New England Journal of Medicine. These cases are known for their diagnostic difficulty and often require collaboration among multiple specialists. MAI-DxO was benchmarked against these cases using a new framework called the Sequential Diagnosis Benchmark (SD Bench). The AI not only achieved higher diagnostic accuracy but also did so 20% more cost-effectively, thanks to its judicious use of diagnostic tests.

MAI-DxO operates by orchestrating multiple advanced AI models: including GPT, Gemini, Claude, LLaMA, and Grok into a virtual panel of expert doctors. This ensemble approach allows the system to simulate collaborative medical reasoning, enhancing both accuracy and transparency. Microsoft emphasizes that the AI is not intended to replace doctors but to augment their capabilities, especially in complex or resource-constrained environments.

Despite the promising results, Microsoft acknowledges limitations. The human doctors in the study were not allowed to use reference tools, which they typically rely on in practice. Moreover, the AI’s performance on more routine medical cases remains to be tested. Regulatory approval and real-world clinical trials are still needed before MAI-DxO can be widely deployed.

Still, Microsoft sees this as a major step toward what it calls “medical superintelligence” a future where AI can democratize access to expert-level diagnostics globally. The company is now working with healthcare providers to test the system in clinical settings, aiming to ensure safety, trust, and effectiveness before broader rollout

(Rh/Sakshi Thakar/MSM/SE)