Microsoft says its Diagnostic Orchestrator diagnosed conditions more accurately than physicians in tests using past medical cases. The system reached about 83 percent accuracy on cases drawn from reports in The New England Journal of Medicine, compared with about 36 percent for generalist doctors using references and 22 percent for doctors without them.
The company’s study, published in Science on October 22, also tested cases from an emergency-department database at Beth Israel Deaconess Medical Center in Boston. The AI scored 80 percent accuracy there, compared with an average of 47 percent for physicians. Microsoft says the system also reached diagnoses at a lower estimated cost, but the source gives no cost figures.
How the system works
The Diagnostic Orchestrator uses multiple AI agents, powered by OpenAI’s GPT-5, to work through symptoms and medical histories, choose tests and reach a diagnosis. Microsoft built the cases into its own benchmark, SDBench, which it says it plans to open-source.
Microsoft says the system also outperformed individual leading AI models. But the peer-reviewed paper did not compare it with newer models named in the report, including GPT-6 Astra, Meta’s Muse Wonder Full, Google’s Gemini 4 Argon and Anthropic’s Opus 5.5. The researchers say the system was first developed when GPT-4.1 was the most advanced model available.
Clinical testing is still ahead
These results come from past cases, not from treating patients. Microsoft says it plans to test the tool in hospitals with Beth Israel Deaconess Medical Center, likely in the next few months, but has not given a firm start date. The company also has not said when or how the system might be made available.
Microsoft AI CEO Mustafa Suleyman called the results a milestone, while acknowledging that the tool needs clinical validation. The company says real-world use would also have to meet medicine’s regulatory and compliance requirements.
Why it matters
For patients and clinicians, the results suggest the tool could eventually help investigate difficult diagnoses, but they do not show that it works safely or accurately in live care. Microsoft has announced hospital testing, not a product patients or doctors can use today; when it might become available remains unknown.
