Microsoft’s experimental diagnostic AI correctly identified 83% of complex medical cases in a simulation, compared with 36% for generalist physicians working individually, in a study published October 22 in Science. The results support the company’s push into clinical decision support, but do not establish that the system outperforms specialist teams or improves care for patients.
The system, called MAI-DxO, coordinates multiple AI agents in a process resembling a medical consultation. The agents gather information, request tests and consider competing diagnoses. Microsoft AI chief executive Mustafa Suleyman described the work as progress toward his ambition of medical superintelligence.
The evaluation used difficult cases from the New England Journal of Medicine. Primary care physicians and general internists could consult reference materials, but worked alone. That comparison is an important limitation: cases of this complexity would ordinarily involve specialists or teams of clinicians, neither of which formed part of the physician benchmark.
The technical results also distinguish the capabilities of the underlying language models from the benefits of Microsoft’s orchestration layer. Researchers tested MAI-DxO with models from OpenAI, Google, Anthropic, Meta and DeepSeek. The strongest results came from OpenAI’s GPT-5 and GPT-5.5, while the additional benefit from orchestration varied substantially by model and type of case.
On complex cases, GPT-5 using a basic prompt achieved 78.9% accuracy. Adding MAI-DxO increased that figure by roughly four percentage points. Microsoft’s estimated per-case cost fell by about $3,000 from approximately $8,400. On emergency department cases, however, the orchestration system produced little improvement in either accuracy or estimated cost.
GPT-5.5 showed a different pattern. MAI-DxO raised its complex-case accuracy from 72% to nearly 87%, at a slightly lower estimated cost. Yet on emergency department cases, both the standalone model and the orchestrated version scored about 85%, with similar costs. Those findings suggest the gains do not transfer uniformly across clinical tasks.
Less capable models sometimes benefited more from the multi-agent approach. Gemini 2.5 Flash improved from 46.4% accuracy on its own to nearly 69% with MAI-DxO. Even so, differences between weaker and stronger underlying models exceeded the improvement supplied by orchestration. Microsoft’s researchers argue that additional accuracy gains become harder as scores approach their upper limit.
The next stage will move beyond simulated cases. Microsoft is partnering with Beth Israel Deaconess Medical Center in Boston to evaluate MAI-DxO’s reasoning in emergency and urgent care settings. Patients will continue receiving standard care. Those who consent will also interact with the AI in parallel, allowing researchers to compare its recommendations with those of treating clinicians.
That design separates evaluation from handing clinical responsibility to the system. Diagnostic accuracy in a case-based test is not the same as demonstrating safe treatment decisions or better patient outcomes across a hospital’s varied population.
Suleyman also told TechCrunch that Microsoft is working with the Mayo Clinic on a healthcare foundation model, with a release planned for 2027. He expects future systems to help clinicians anticipate how a patient’s medical record may develop, although that remains a development goal rather than a demonstrated outcome of the Science study.
Microsoft already supplies clinicians with dictation and radiology tools. Its broader research agenda aims to extend beyond isolated diagnostic benchmarks into reasoning and coordination throughout care. The planned hospital evaluations will test how MAI-DxO’s performance compares with clinicians’ decisions under conditions that a simulation cannot fully reproduce.
Sources: TechCrunch AI