A Microsoft study in Science reports a large accuracy gap between its diagnostic AI system and non-specialist physicians on complex cases.
TechCrunch reports that Microsoft AI, led by Mustafa Suleyman, published a study in the journal Science showing its orchestration system, MAI DxO, correctly diagnosed complex cases drawn from the New England Journal of Medicine 83% of the time in a simulation. Primary care doctors and general internists working alone with reference materials were correct 36% of the time. The comparison did not include specialists or physician teams, who typically handle cases of this complexity.
Suleyman said much of the improvement traces to the strength of the underlying AI models themselves, with Microsoft's orchestration layer adding further gains on top, though the system showed little improvement over the base model on emergency department cases. Microsoft plans a follow-up study with Beth Israel Deaconess Medical Center to compare the system's recommendations against real clinical decisions, and said it is working with the Mayo Clinic to train a foundation model for healthcare, due for release next year.
Why it matters: a strong result in simulation is not the same as a validated clinical tool, and enterprises piloting AI diagnostic or decision support systems in healthcare need independent, real-world evidence, not vendor-run benchmarks, before trusting them with patient care decisions.