Microsoft AI beats doctors in study

Microsoft’s AI division has unveiled a diagnostic system that correctly identified complex medical conditions in simulations at more than double the rate of human generalists. The system, known as MAI-DxO, coordinates multiple artificial intelligence agents to mimic a panel of specialists who question patients, order tests, and debate potential diagnoses. According to a study published in Science on Thursday, the technology achieved an 83% accuracy rate on difficult cases drawn from the New England Journal of Medicine. In comparison, primary care doctors and general internists working individually with reference materials were correct only 36% of the time.

The path to this milestone involves Microsoft testing the orchestration framework against models from various providers, including OpenAI, Google, Anthropic, Meta, and DeepSeek. The most significant results were observed when the system utilized OpenAI’s GPT-5 and GPT-5.5 models. While the underlying power of these models drove much of the improvement, Microsoft’s coordination layer added further gains, particularly on complex cases. For instance, a weaker model called Gemini 2.5 Flash improved from 46.4% to nearly 69% accuracy when assisted by the system. However, the technology showed less impact on emergency department cases, where baseline accuracy was already high and costs remained similar.

Cost efficiency is a key metric in the study. When using GPT-5 with a basic prompt, the system cost approximately $8,400 per case. The orchestration layer reduced this expense by about $3,000 while increasing accuracy by four points. Mustafa Suleyman, CEO of Microsoft AI, views these results as a critical step toward "medical superintelligence," a concept defined as surpassing contemporary human-led health systems in reasoning and coordination. He noted that the company is currently training a foundation model for healthcare in partnership with the Mayo Clinic, with a release planned for next year.

Suleyman, who previously co-founded DeepMind, believes the next 24 to 36 months will see an order-of-magnitude increase in the capability of healthcare models. He expects a "coding style breakthrough" for medical records in 2027, allowing for highly accurate predictions of patient outcomes. To validate these claims in real-world settings, Microsoft is launching a major evaluation in partnership with Beth Israel Deaconess Medical Center in Boston. This study will compare the AI’s recommendations against human doctors in both emergency and urgent care environments, aiming to provide compelling evidence for broader deployment.

This move places Microsoft in direct competition with Google and OpenAI, which have also claimed their AI can match or exceed human doctors. Google recently published a study in The Lancet evaluating its AMIE diagnostic AI in real-world urgent care visits. Microsoft aims to integrate its tools beyond individual tasks, such as answering exam questions, into the broader clinical workflow. Dominic King, vice president of Health at Microsoft AI, stated that the technology is ready for these real-world evaluations, noting the diverse challenges presented by a Boston emergency department on a busy evening. The company sees this as a differentiator in the race among frontier AI labs.

Microsoft already sells dictation and radiology tools to clinicians and positions the new diagnostic system as an expansion of these efforts. Suleyman emphasized the company’s commitment to building state-of-the-art clinical decision support products. He believes the rapid progress seen in coding and chat interfaces will soon replicate in medical records and clinical decision support as the "flywheel" of real-world usage begins to turn. The goal is to provide clinicians with foresight that acts as "gold dust" in their daily practice, ultimately addressing the growing mismatch between the supply of medical expertise and patient demand.