# MDIA: A Multi-Agent Diagnostic Intelligence Pipeline on HealthBench Professional

Source: [MDIA: A Multi-Agent Diagnostic Intelligence Pipeline on HealthBench Professional](<https://www.sota2.com/research/paper/6a152939dbd7150506f7e098>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-27T05:00:05.521Z

Authors: Roberto Cruz, David Rey-Blanco

Published: 2026-05-23T18:36:58.000Z

## Abstract

Most reported gains on agentic-LLM clinical benchmarks are often attributed to prompt engineering, yet our results suggest that larger improvements can come from architectural and engine-level design. We present MDIA, a Multi-agent Diagnostic Intelligence Agent implemented as a 7-node specialty-routed clinical reasoning graph, on the full HealthBench Professional benchmark (n = 525), on a non-fine-tuned LLM. MDIA achieves 0.6272 under OpenAI's GPT-5.4-2026-03-05, which is +3.72 pp above the performance of OpenAI's ChatGPT for Clinicians. The experimental work shows that performance lift is attributable to system architecture: specialty routing, multi-turn context preservation, drug-state safety gating, site-filtered search, length-aware synthesis, and engine-level reliability. These findings support the view that agentic clinical benchmark performance is shaped both by the underlying foundation model and the orchestration architecture. Nevertheless, we also noticed notable differences when using other models as a grader; in particular, when using Gemini 2.5 Pro, MDIA scored 0.6585, which suggests that the choice of grader is a source of variability. Robust evaluation of LLMs would therefore require assessment across several independent grader models.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Clinical Reasoning on HealthBench Professional (525 cases)](<https://www.sota2.com/research/sota/clinical-reasoning-on-healthbench-professional-525-cases.md>): Clinical Reasoning; HealthBench Professional (525 cases); Overall Score: 62.72.

- [Medical Question Answering on HealthBench Professional](<https://www.sota2.com/research/sota/medical-question-answering-on-healthbench-professional.md>): Medical Question Answering; HealthBench Professional; Score: 62.7.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2605.24699>)

[PDF](<https://arxiv.org/pdf/2605.24699.pdf>)