Medical Calculation on MedCalc-Bench
81.4AccuracyGPT-5
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GPT-5Family=Frontier API2026.05 | 81.4 | — | — | — | — | — | — | |
| Claude Sonnet 4.5Backbone=Claude Sonnet 4.52026.07 | 76 | — | — | — | — | — | — | |
| DeepSeek-R1Backbone=DeepSeek-R12026.07 | 72.7 | — | — | — | — | — | — | |
| GPT-4.1Family=Frontier API2026.05 | 72 | — | — | — | — | — | — | |
| GPT-4.1-miniFamily=Frontier API2026.05 | 67.5 | — | — | — | — | — | — | |
| FaithMed (Qwen3 4B)Backbone=Qwen3 4B2026.07 | 56.5 | — | — | — | — | — | — | |
| MedGemma-27BFamily=Medical / open-source2026.05 | 51.9 | — | — | — | — | — | — | |
| MedGemma 27BBackbone=MedGemma 27B2026.07 | 49 | — | — | — | — | — | — | |
| CLR-voyance-8BFamily=8B family, Merge Strategy=DELLA-Linear2026.05 | 46 | — | — | — | — | — | — | |
| Qwen3 8BBackbone=Qwen3 8B2026.07 | 44.9 | — | — | — | — | — | — | |
| Qwen3-8B baseFamily=8B family2026.05 | 41.5 | — | — | — | — | — | — | |
| Agentic Search + GRPO (Qwen3 4B)Backbone=Qwen3 4B2026.07 | 39 | — | — | — | — | — | — | |
| FaithMed (Qwen3 1.7B)Backbone=Qwen3 1.7B2026.07 | 33.2 | — | — | — | — | — | — | |
| DS-R1-0528Family=Medical / open-source2026.05 | 31.6 | — | — | — | — | — | — | |
| Agentic Search (Qwen3 4B)Backbone=Qwen3 4B2026.07 | 29.6 | — | — | — | — | — | — | |
| Agentic Search + GRPO (Qwen3 1.7B)Backbone=Qwen3 1.7B2026.07 | 27.1 | — | — | — | — | — | — | |
| HuatuoGPT-o1Family=Medical / open-source2026.05 | 27 | — | — | — | — | — | — | |
| Gemma 3 12BBackbone=Gemma 3 12B2026.07 | 27 | — | — | — | — | — | — | |
| Llama 3.1 8B InstructBackbone=Llama 3.1 8B2026.07 | 21.4 | — | — | — | — | — | — | |
| MG-BFamily=4B family, Merge Strategy=Breadcrumbs2026.05 | 21 | — | — | — | — | — | — | |
| MG-B+AFamily=4B family, Merge Strategy=Breadcrumbs+AIM2026.05 | 18.5 | — | — | — | — | — | — | |
| Base (Qwen3 1.7B)Backbone=Qwen3 1.7B2026.07 | 16.9 | — | — | — | — | — | — | |
| MG-Δ+AFamily=4B family, Merge Strategy=DELLA-Linear+AIM2026.05 | 15.5 | — | — | — | — | — | — | |
| Agentic Search (Qwen3 1.7B)Backbone=Qwen3 1.7B2026.07 | 15.3 | — | — | — | — | — | — | |
| CLR-voyance-4BFamily=4B family, Merge Strategy=DELLA-Linear2026.05 | 15 | — | — | — | — | — | — | |
| Base (Qwen3 4B)Backbone=Qwen3 4B2026.07 | 13.7 | — | — | — | — | — | — | |
| MG-4B baseFamily=4B family2026.05 | 10.7 | — | — | — | — | — | — | |
| MedRAG (Qwen3 4B)Backbone=Qwen3 4B2026.07 | 10.3 | — | — | — | — | — | — | |
| DS-R1-DistillFamily=Medical / open-source2026.05 | 8.6 | — | — | — | — | — | — | |
| MedRAG (Qwen3 1.7B)Backbone=Qwen3 1.7B2026.07 | 8.4 | — | — | — | — | — | — | |
| Gemma-3-4BFamily=Medical / open-source2026.05 | 7.3 | — | — | — | — | — | — | |
| Default (Greedy) CoTMetric=Exact-Match2026.01 | — | 0.14 | 0.24 | 0.43 | 0.38 | 0.41 | 0.32 | |
| Default (Sampled) CoTMetric=Exact-Match2026.01 | — | 0.16 | 0.29 | 0.45 | 0.38 | 0.35 | 0.33 | |
| Empty CoTMetric=Exact-Match2026.01 | — | 0.1 | 0.18 | 0.45 | 0.36 | 0.38 | 0.29 | |
| Ens. (QwQ+DAPO/OSS)Metric=Exact-Match2026.01 | — | 0.39 | 0.37 | 0.41 | 0.41 | 0.4 | 0.4 | |
| Ens. (QwQ+OpenT/OSS)Metric=Exact-Match2026.01 | — | 0.34 | 0.41 | 0.42 | 0.38 | 0.4 | 0.39 | |
| Ens. (QwQ+OSS/DAPO)Metric=Exact-Match2026.01 | — | 0.28 | 0.37 | 0.45 | 0.42 | 0.43 | 0.38 | |
| Trans. CoT (DAPO)Metric=Exact-Match2026.01 | — | 0.31 | 0.4 | 0.39 | 0.4 | 0.41 | 0.38 | |
| Trans. CoT (NRR)Metric=Exact-Match2026.01 | — | 0.14 | 0.15 | 0.24 | 0.3 | 0.34 | 0.23 | |
| Trans. CoT (OpenT)Metric=Exact-Match2026.01 | — | 0.21 | 0.24 | 0.26 | 0.26 | 0.25 | 0.24 | |
| Trans. CoT (OSS)Metric=Exact-Match2026.01 | — | 0.26 | 0.44 | 0.43 | 0.4 | 0.44 | 0.39 | |
| Trans. CoT (QwQ)Metric=Exact-Match2026.01 | — | 0.34 | 0.37 | 0.39 | 0.38 | 0.37 | 0.37 |