Medical Decision-Making on MedQA
92.59AccuracyMAC
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| MACType=Closed-source, Variant=w/ GPTs2025.07 | 92.59 | 92.59 | 92.62 | 98.14 | 90.71 | |
| GPT-5.1Type=Closed-source2025.07 | 91.04 | 91.08 | 91.18 | 98.11 | 88.79 | |
| Gemini 2.5 FlashType=Closed-source2025.07 | 90.18 | 90.18 | 90.21 | 97.53 | 87.69 | |
| MACType=Open-access2025.07 | 79.94 | 80.24 | 80.66 | 97.01 | 74.94 | |
| DebateType=Open-access2025.07 | 79.44 | 79.47 | 80.38 | 94.83 | 74.39 | |
| QWQ 32BType=Open-access2025.07 | 78.76 | 79.05 | 79.43 | 96.17 | 73.45 | |
| Openthinker 32BType=Open-access2025.07 | 77.63 | 77.96 | 78.42 | 96 | 72.05 | |
| Deepseek-R1 32BType=Open-access2025.07 | 76.98 | 77.7 | 78.59 | 96.07 | 71.35 | |
| Phi4 14BType=Open-access2025.07 | 75.15 | 75.24 | 75.42 | 95.08 | 68.88 | |
| Claude Haiku 4.5Type=Closed-source2025.07 | 71.93 | 72.2 | 73.22 | 94.21 | 65.14 | |
| Llama3 instruct 70BType=Open-access2025.07 | 71.41 | 71.37 | 71.49 | 93.22 | 64.18 | |
| WizardLM 8x22 141BType=Open-access2025.07 | 70.04 | 71.67 | 73.62 | 94.55 | 63.08 | |
| Qwen2.5 32BType=Open-access2025.07 | 66.11 | 66.27 | 66.99 | 93.9 | 57.71 | |
| Qwen2.5 14BType=Open-access2025.07 | 64.55 | 64.6 | 64.93 | 92.14 | 55.67 | |
| Mixtral 8x22 141BType=Open-access2025.07 | 60.88 | 63.12 | 65.85 | 93.67 | 52.12 | |
| Qwen1.5 Chat 110BType=Open-access2025.07 | 58.42 | 60.87 | 63.88 | 93.57 | 49.24 | |
| MoAType=Open-access2025.07 | 57.06 | 61.27 | 67.17 | 93.22 | 48.65 | |
| Med42 8BType=Open-access2025.07 | 55.96 | 55.91 | 56.13 | 90.22 | 44.78 | |
| dbrx-instruct 132BType=Open-access2025.07 | 51.2 | 52.67 | 56.04 | 90.08 | 40.24 | |
| SelfMoAType=Open-access2025.07 | 47.96 | 55.64 | 71.66 | 92.7 | 42.19 | |
| Medllama2 7BType=Open-access2025.07 | 29.98 | 31.36 | 36.31 | 88.81 | 16.65 | |
| Qwen1.5 Chat 72BType=Open-access2025.07 | 29.51 | 34.94 | 50.93 | 90.22 | 23.45 | |
| HuatuoGPT-o1 8BType=Open-access2025.07 | 27.09 | 31.44 | 55.11 | 89.98 | 24.19 |