Average Final Diagnosis Prediction on MIMIC, MedCaseReasoning, and ER-Reason
62.1Average AccuracyCounterfactual Multi-agent Diagnostic Framework
Evaluation results
| Method | Links | |
|---|---|---|
| Counterfactual Multi-agent Diagnostic FrameworkModel=GPT-4o mini, Setting=Agents2026.03 | 62.1 | |
| MDAgentsModel=GPT-4o mini, Setting=Agents2026.03 | 58.1 | |
| Counterfactual Multi-agent Diagnostic FrameworkModel=Deepseek R1, Setting=Agents2026.03 | 55.2 | |
| FSModel=GPT-4o mini, Setting=Prompting2026.03 | 52 | |
| FS CoTModel=GPT-4o mini, Setting=Prompting2026.03 | 52 | |
| FS CoTModel=Deepseek R1, Setting=Prompting2026.03 | 51.1 | |
| MACModel=GPT-4o mini, Setting=Agents2026.03 | 51.1 | |
| ZSModel=GPT-4o mini, Setting=Prompting2026.03 | 50.7 | |
| MedAgentsModel=GPT-4o mini, Setting=Agents2026.03 | 50.2 | |
| ZS CoTModel=Deepseek R1, Setting=Prompting2026.03 | 48.7 | |
| FSModel=Deepseek R1, Setting=Prompting2026.03 | 48.6 | |
| MedAgentsModel=Deepseek R1, Setting=Agents2026.03 | 48.6 | |
| ZS CoTModel=GPT-4o mini, Setting=Prompting2026.03 | 48.5 | |
| MDAgentsModel=Deepseek R1, Setting=Agents2026.03 | 46.7 | |
| ZSModel=Deepseek R1, Setting=Prompting2026.03 | 43.9 | |
| MACModel=Deepseek R1, Setting=Agents2026.03 | 42.9 | |
| Counterfactual Multi-agent Diagnostic FrameworkModel=medgemma, Setting=Agents2026.03 | 40.9 | |
| Counterfactual Multi-agent Diagnostic FrameworkModel=Qwen, Setting=Agents2026.03 | 40.2 | |
| Counterfactual Multi-agent Diagnostic FrameworkModel=Llama, Setting=Agents2026.03 | 39 | |
| ZS CoTModel=Qwen, Setting=Prompting2026.03 | 37.6 | |
| FSModel=Qwen, Setting=Prompting2026.03 | 37.5 | |
| ZSModel=Qwen, Setting=Prompting2026.03 | 37.1 | |
| FS CoTModel=Qwen, Setting=Prompting2026.03 | 37.1 | |
| Counterfactual Multi-agent Diagnostic FrameworkModel=m1-7B-23k, Setting=Agents2026.03 | 36.8 | |
| MedAgentsModel=Qwen, Setting=Agents2026.03 | 36.6 | |
| FSModel=medgemma, Setting=Prompting2026.03 | 36.5 | |
| MDAgentsModel=Qwen, Setting=Agents2026.03 | 35.8 | |
| MACModel=Qwen, Setting=Agents2026.03 | 35.5 | |
| FSModel=Llama, Setting=Prompting2026.03 | 35.4 | |
| ZSModel=m1-7B-23k, Setting=Prompting2026.03 | 35.2 | |
| MDAgentsModel=Llama, Setting=Agents2026.03 | 35 | |
| ZS CoTModel=m1-7B-23k, Setting=Prompting2026.03 | 34.6 | |
| ZS CoTModel=medgemma, Setting=Prompting2026.03 | 34.5 | |
| FS CoTModel=medgemma, Setting=Prompting2026.03 | 34.4 | |
| FS CoTModel=Llama, Setting=Prompting2026.03 | 34.1 | |
| ZSModel=medgemma, Setting=Prompting2026.03 | 34.1 | |
| MACModel=medgemma, Setting=Agents2026.03 | 33.3 | |
| MDAgentsModel=m1-7B-23k, Setting=Agents2026.03 | 33.1 | |
| MedAgentsModel=Llama, Setting=Agents2026.03 | 32.9 | |
| MDAgentsModel=medgemma, Setting=Agents2026.03 | 32.5 | |
| FS CoTModel=m1-7B-23k, Setting=Prompting2026.03 | 31.5 | |
| Counterfactual Multi-agent Diagnostic FrameworkModel=MedReason, Setting=Agents2026.03 | 31.5 | |
| FSModel=m1-7B-23k, Setting=Prompting2026.03 | 31 | |
| MedAgentsModel=medgemma, Setting=Agents2026.03 | 30.9 | |
| MedAgentsModel=MedReason, Setting=Agents2026.03 | 29.3 | |
| MACModel=Llama, Setting=Agents2026.03 | 29 | |
| MACModel=MedReason, Setting=Agents2026.03 | 27.7 | |
| ZSModel=Llama, Setting=Prompting2026.03 | 27.3 | |
| FS CoTModel=MedReason, Setting=Prompting2026.03 | 27.3 | |
| MDAgentsModel=MedReason, Setting=Agents2026.03 | 26.9 | |
| ZS CoTModel=Llama, Setting=Prompting2026.03 | 25.8 | |
| ZSModel=MedReason, Setting=Prompting2026.03 | 25.5 | |
| MedAgentsModel=m1-7B-23k, Setting=Agents2026.03 | 25.3 | |
| MACModel=m1-7B-23k, Setting=Agents2026.03 | 24.5 | |
| ZS CoTModel=MedReason, Setting=Prompting2026.03 | 24.4 | |
| FSModel=MedReason, Setting=Prompting2026.03 | 23.4 |