Microsoft reported that Microsoft medical AI diagnosed complex conditions with higher accuracy than human doctors during benchmark tests published in Science on Thursday. Known as the AI-powered Microsoft AI Diagnostic Orchestrator (MAI-DxO), the system posted an 83.2% accuracy score on difficult clinical cases from The New England Journal of Medicine, while generalist doctors with reference books reached 36.1% [1]. Microsoft AI CEO Mustafa Suleyman told The Verge that the company aims to match human medical performance over the next few years [3]. The work first appeared on arXiv as an unreviewed preprint before passing peer review for journal publication [2].
Accuracy Scores on Hospital and Journal Cases
The benchmark test examined how doctors and language models handle complex patient files drawn from The New England Journal of Medicine clinicopathological conferences [1]. In those tests, MAI-DxO reached 83.2% diagnostic accuracy, beating working generalist doctors who had access to medical books and online notes. When doctors took the same diagnostic tests without reference books, their accuracy fell to roughly 22%, which left the AI tool nearly four times as accurate as unaided staff on the same clinical conference files [2]. Study authors H. Nori, M. Daswani, and C. Kelly tracked sequential reasoning over hundreds of published case reports to check how clinical choices change as new patient clues emerge during a workup. Clues shifted with each test [1].
A second test checked emergency department admissions from Beth Israel Deaconess Medical Center in Boston, Massachusetts, using the MIMIC-IV-ED database [1]. MAI-DxO achieved 80.3% accuracy on these acute hospital files, compared with 46.8% for generalist doctors, representing an average of 47 percent in rounded figures [2, 3]. Doctor scores showed a narrower margin on emergency files than on journal case conferences [3]. Emergency room files present diverse patient signs, and the tool maintained steady scores across urgent admissions [1].
Microsoft built the system using OpenAI language models, starting work when GPT-4.1 was the newest model available [3]. The tool posted its top scores with GPT-5, and later experiments showed further gains with GPT-5.5 [2]. The peer-reviewed paper did not test newer frontier models like GPT-6 Astra, Meta Muse Wonder Full, Google Gemini 4 Argon, or Anthropic Opus 5.5. Newer models were excluded [3].
How MAI-DxO Mimics a Doctor Panel
The authors noted that earlier evaluations failed to measure LLMs’ sequential diagnostic reasoning in clinical setups. Earlier AI benchmarks tested models by presenting a full medical history, symptom list, and lab report in one prompt [2]. In contrast, sequential diagnosis starts with a chief complaint, and the model must ask follow-up questions and pick tests before naming an illness [1]. That step-by-step clinical logic differs from long-range risk modeling where AI models predict intimate partner violence years in advance, because MAI-DxO focuses on immediate patient presentations. Because doctors collect patient clues in stages, this setup mirrors real office visits [2]. It stops models from guessing a condition before gathering basic evidence [1].
The platform acts like a panel of medical specialists by assigning separate roles to multiple AI agents [1]. One agent suggests possible diagnoses, another selects medical tests, a third challenges assumptions, and a fourth tracks testing costs [2]. Microsoft plans to open-source its benchmark, named the Sequential Diagnosis Benchmark or SDBench, so outside labs can run tests on the same cases [3].
The multi-agent setup helps counter cognitive bias, such as anchoring on a first guess [2]. In the study, the authors explain that Microsoft medical AI “simulates a panel of physicians, continually revising the differential diagnosis and selecting cost-effective tests”. Coordinated agents question weak clinical assumptions [1].
Simulated Costs and Workup Tests
Beyond accuracy gains, Microsoft said its software reached diagnoses with fewer medical steps than generalist doctors. The company stated in a blog post: “It also manages to do this at half or even one third of the cost, getting to the right diagnosis with fewer tests and procedures.” But these cost figures come from estimated dollar values attached to simulated patient cases rather than hospital billing receipts. Real hospital care involves logistical delays, lab fees, and staffing expenses that simulated files do not capture, meaning that true savings will remain unclear until hospitals test the tool in live clinics. Costs were simulated [2].
The benchmark compared MAI-DxO with generalist doctors instead of experienced specialists. Coauthors S. Lundberg and M. P. Lungren noted in the study that they “would expect the performance gap to narrow” if the software faced specialist doctors who treat narrow disease groups every day [1, 2]. General doctors in busy clinics face short visit times and broad patient complaints, which can limit how many rare conditions they recall during routine exams [1].
Other teams have seen similar test results in recent trials. In an earlier 2026 triage study run by Harvard University and Beth Israel Deaconess Medical Center, OpenAI’s o1-preview reasoning model beat human doctors at sorting incoming emergency patients [2].
Clinical Trials at Beth Israel Deaconess
Microsoft plans to evaluate the diagnostic software inside live hospital wards to test how it works with working doctors. Testing will take place at Beth Israel Deaconess Medical Center in Boston, Massachusetts, where observers will track how doctors use the tool during patient care [2]. Dr. Dominic King, Microsoft’s AI health lead and study coauthor, told The Verge that hospital tests should start in the next couple of months. The team wants to learn whether model advice helps doctors make better choices when managing busy emergency shifts and heavy patient loads [3].
Commercialization could develop in future product updates, although Microsoft has not disclosed retail pricing or release schedules. Dr. Dominic King and Microsoft AI CEO Mustafa Suleyman did not give a firm release date for the tool. King pointed to complex regulatory rules that software makers must satisfy before selling clinical tools to health systems. Microsoft already sells digital health services to hospital groups in the United States, and health questions rank among the most common queries on its Copilot chatbot [3].
Diagnosis is one part of Microsoft’s wider healthcare vision, described as medical superintelligence. Microsoft began using the term superintelligence before Donald Trump and his administration co-opted it as a catch-all word for AI. In BMJ Digital Health & AI, company researchers outlined five other goals for clinical AI, including care coordination, treatment planning, and patient communication [3].
Microsoft Medical AI Encounters Practical Limits
Medical specialists caution that high benchmark scores do not mean AI tools can replace licensed doctors in exam rooms. Microsoft says its software is meant to assist human clinicians rather than substitute for them during patient evaluations [2]. Suleyman said he wants the tool to remove “the drudgery” from medical practice so doctors can spend more time talking with their patients. He stated: “We’re going to build something that is indistinguishable from human performance … in the next couple of years.” Doctors retain final responsibility [3].
Thorny legal liability questions emerge when an AI-assisted diagnosis leads to patient injury. If software suggests an incorrect test step or misses an acute illness, existing health laws do not say whether legal blame falls on the attending doctor, the hospital system, or the software vendor. Until regulators set clear liability rules, hospital managers remain cautious about giving models an independent role in clinical diagnostics [2].
Relying on automated tools may also weaken doctor skills over time. A 2025 study found that doctors who used AI tools to spot cancer risks during colonoscopies scored lower on manual checks once the software was removed. Junior doctors who rely on Microsoft medical AI could lose diagnostic sharpness without regular unaided practice [2]. Still, Mustafa Suleyman expressed optimism about the future of medical technology, telling The Verge: “I think this is why we build technology.” [3]
- ACADEMIC JOURNAL Nori, H., Daswani, M., Kelly, C., Lundberg, S., Ribeiro, M. T., Wilson, M., Liu, X., Sounderajah, V., Carlson, J. M., Lungren, M. P., Gross, B., Hames, P., Suleyman, M., King, D., & Horvitz, E. (2026). Sequential diagnosis with language models. Science, 394(6822), 423-428. [Article Link]
- ONLINE NEWS Yildirim, E. (2026, October 22). Microsoft says its new AI is more than twice as good at diagnosing diseases as doctors. Gizmodo. [Article Link]
- ONLINE NEWS Hart, R. (2026, October 22). Microsoft says its AI tool is better than doctors at diagnosing diseases. The Verge. [Article Link]
