Medical Reasoning on MMLU Pro
80.7AccuracyMDAgents
Evaluation Results
| Method | Links | |
|---|---|---|
| MDAgentsBase Model=GPT-4o2025.08 | 80.7 | |
| DyLANBase Model=GPT-4o2025.08 | 80 | |
| TMA-AllComponBase Model=GPT-4o2025.08 | 79.7 | |
| ReConcileBase Model=GPT-4o2025.08 | 78.8 | |
| m1kModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 64.5 | |
| Middle PerplexityModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 63.6 | |
| OursModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 63 | |
| RandomModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 62.7 | |
| Embedding DiversityModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 62.5 | |
| S2LModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 62.5 | |
| LearnabilityModel=Qwen2.5-7B-Instruct, Training=Finetuning, Sample size=1000 examples2026.06 | 60.8 | |
| MedAgentsBase Model=GPT-4o2025.08 | 46.7 | |
| TMA-AllComponBase Model=MedGemma-4B2025.08 | 46.4 | |
| ReConcileBase Model=Gemma-3-4B2025.08 | 35.8 | |
| TMA-AllComponBase Model=Gemma-3-4B2025.08 | 35.6 | |
| DyLANBase Model=Gemma-3-4B2025.08 | 27.4 | |
| MDAgentsBase Model=Gemma-3-4B2025.08 | 24.4 | |
| MedAgentsBase Model=Gemma-3-4B2025.08 | 23.6 |