Medical Question Answering on MedQA-USMLE (letter exact-match)
79.8Accuracy (Letter Match)MAGE
Evaluation results
| Method | Links | |
|---|---|---|
| MAGEBackbone=Llama-3.1-8B, Execution Scaffold=Opus, Number of seeds=52026.05 | 79.8 | |
| MAGEBackbone=Llama-3.1-8B, Execution Scaffold=Sonnet, Number of seeds=52026.05 | 74.6 | |
| MedpromptBackbone=Llama-3.1-8B2026.05 | 74 | |
| 0-shot CoTBackbone=Llama-3.1-8B, Evaluation Protocol=0-shot2026.05 | 64.5 |