Multiple Choice Question Answering on MMLU ProX Health
76.56Accuracy (ZH)MED-COREASONER
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| MED-COREASONERModel Category=Closed-Source, Backbone=GPT-5.12026.01 | 76.56 | 77.87 | 76.85 | 77.72 | 77.43 | 78.6 | 78.31 | 75.98 | 77.42 | |
| GPT-5.2Model Category=Closed-Source2026.01 | 72.63 | 73.94 | 73.94 | 76.42 | 77 | 75.69 | 75.98 | 70.6 | 74.53 | |
| MED-COREASONERModel Category=Open-Source, Backbone=DeepSeek-3.22026.01 | 71.91 | 71.91 | 72.34 | 73.36 | 74.52 | 73.22 | 73.91 | 68.85 | 72.5 | |
| MED-COREASONERModel Category=Closed-Source, Backbone=GPT-4o2026.01 | 71.9 | 71.32 | 71.62 | 71.76 | 73.22 | 73.21 | 73.07 | 69.14 | 71.91 | |
| GPT-5.1Model Category=Closed-Source2026.01 | 71.32 | 70.45 | 70.45 | 71.32 | 72.78 | 72.63 | 74.24 | 67.89 | 71.39 | |
| RAG + CoTModel Category=Closed-Source, Backbone=GPT-5.1, Reasoning Strategy=RAG + Chain-of-Thought2026.01 | 69.14 | 69.87 | 67.83 | 70.31 | 72.63 | 70.89 | 72.63 | 65.6 | 69.86 | |
| Self-ConsistencyModel Category=Closed-Source, Backbone=GPT-5.1, Reasoning Strategy=Self-Consistency2026.01 | 68.08 | 68.51 | 66.96 | 70.12 | 71.18 | 71.72 | 71.97 | 62.39 | 68.87 | |
| GPT-4oModel Category=Closed-Source2026.01 | 67.39 | 67.39 | 65.94 | 69.14 | 69.72 | 70.6 | 71.47 | 61.86 | 67.94 | |
| CoTModel Category=Closed-Source, Backbone=GPT-5.1, Reasoning Strategy=Chain-of-Thought2026.01 | 67.1 | 61.28 | 63.61 | 70.31 | 72.34 | 70.01 | 71.32 | 60.99 | 67.12 | |
| DeepSeek-3.2Model Category=Open-Source2026.01 | 66.81 | 66.81 | 61.28 | 69 | 68.7 | 67.89 | 69 | 51.67 | 65.15 | |
| SoTModel Category=Closed-Source, Backbone=GPT-5.1, Reasoning Strategy=Structured-of-Thought2026.01 | 65.21 | 64.63 | 62.45 | 67.69 | 68.85 | 68.27 | 69.72 | 56.19 | 65.37 | |
| MED-COREASONERModel Category=Open-Source, Backbone=Qwen3-30B2026.01 | 64.92 | 62.15 | 59.39 | 65.65 | 66.38 | 67.83 | 68.41 | 39.01 | 61.72 | |
| Qwen3-30BModel Category=Open-Source2026.01 | 61.86 | 56.91 | 54.15 | 61.72 | 61.72 | 64.48 | 63.32 | 27.22 | 56.42 | |
| Qwen2.5-72BModel Category=Open-Source2026.01 | 59.39 | 56.48 | 54.15 | 59.83 | 61.86 | 61.51 | 60.26 | 30.13 | 55.45 | |
| Qwen2.5-32BModel Category=Open-Source2026.01 | 58.22 | 54.73 | 48.33 | 59.1 | 58.66 | 60.41 | 58.95 | 26.76 | 53.15 | |
| Claude-3.5-haikuModel Category=Closed-Source2026.01 | 56.33 | 57.64 | 54.88 | 60.7 | 61.72 | 59.97 | 62.59 | 35.81 | 56.21 | |
| LLaMA3.1-70BModel Category=Open-Source2026.01 | 51.09 | 48.33 | 47.45 | 59.53 | 60.84 | 60.26 | 60.84 | 44.25 | 54.07 |