Health-related dialogue and decision-making on HealthBench Main
46.38Average ScoreGPT-5
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-5Model Category=Proprietary Models2025.10 | 46.38 | 63.22 | 34.8 | 57.12 | 37.12 | 40.16 | 37.84 | 54.26 | 44.68 | 62.5 | 61.31 | 53.32 | 50.02 | |
| DOCTOR-R12025.10 | 36.29 | 54.44 | 29.17 | 47.16 | 24.74 | 33.71 | 26.39 | 34.25 | 37.84 | 64.15 | 54.39 | 49.24 | 40.93 | |
| SEMA-RAGBackbone=deepseek-v3.12026.05 | 33.64 | — | — | — | — | — | — | — | 37.22 | — | 47.58 | — | 38.3 | |
| Baichuan-M2-32BModel Category=Open-Source Models, Scale=>=32B2025.10 | 33.16 | 20.16 | 8.9 | 22.11 | 18.4 | 17.31 | 28.24 | 24.45 | 33.95 | 58.01 | 52.4 | 46.8 | 40.03 | |
| Grok-4Model Category=Proprietary Models2025.10 | 33.03 | 12.5 | 21.79 | 24.25 | 19.46 | 18.29 | 20.8 | 16.05 | 37.95 | 61.35 | 48.55 | 45.62 | 34.84 | |
| DOCTOR-R1Ablation=w/o Proc. Reward2025.10 | 32.61 | 51.21 | 21.35 | 39.05 | 22.92 | 32.78 | 25.22 | 32.61 | 34.43 | 59.99 | 47.69 | 46.35 | 38.8 | |
| DOCTOR-R1Ablation=w/o Experience2025.10 | 31.69 | 47.24 | 25.3 | 38.49 | 21.58 | 31.79 | 24.15 | 30.55 | 35.96 | 59.19 | 51.75 | 45.31 | 36.23 | |
| GPT-4.1Model Category=Proprietary Models2025.10 | 31.18 | 53.98 | 23.51 | 37.03 | 22.36 | 29.4 | 21.79 | 45.93 | 34.78 | 60.65 | 54.32 | 44.81 | 34.84 | |
| MedRAGBackbone=deepseek-v3.12026.05 | 26.87 | — | — | — | — | — | — | — | 31.34 | — | 41.66 | — | 30.17 | |
| UltraMedical-70BModel Category=Open-Source Models, Scale=>=32B2025.10 | 26.38 | 38.59 | 23.25 | 31.42 | 19.42 | 18.13 | 16.44 | 45.23 | 32.6 | 50.62 | 41.22 | 45.49 | 27.61 | |
| Med42-v2-70BModel Category=Open-Source Models, Scale=>=32B2025.10 | 26.04 | 41.49 | 24.17 | 35.64 | 13.47 | 20.86 | 15.59 | 33.73 | 30.61 | 56.08 | 51.45 | 41.36 | 26.95 | |
| Claude Sonnet 4Model Category=Proprietary Models2025.10 | 25.69 | 51.54 | 21.87 | 28.6 | 16.52 | 23.42 | 18.38 | 38.23 | 28.78 | 58.37 | 44.59 | 41.81 | 31.65 | |
| Base (Qwen3-8B)2025.10 | 25.13 | 45.42 | 16.5 | 27.98 | 15.26 | 25.34 | 16.42 | 30.69 | 28.57 | 49.35 | 43.51 | 43 | 27.24 | |
| UltraMedical-8BModel Category=Open-Source Models, Scale=7B-8B2025.10 | 22.19 | 46.54 | 19.52 | 26.12 | 13.78 | 19.31 | 18.28 | 26.72 | 25.5 | 57.4 | 44.68 | 40.26 | 27.4 | |
| HuatuoGPT-o1-70BModel Category=Open-Source Models, Scale=>=32B2025.10 | 21.21 | 36.58 | 18.21 | 23.97 | 12.01 | 19.75 | 12.87 | 29.15 | 26.35 | 54.93 | 49.98 | 37.97 | 23.16 | |
| Gemini-2.5-FlashModel Category=Proprietary Models2025.10 | 19.35 | 44.65 | 9.12 | 14.66 | 14.05 | 20.45 | 13.09 | 33.69 | 22.45 | 46.98 | 35.84 | 35.81 | 21.56 | |
| OpenBioLLM-70BModel Category=Open-Source Models, Scale=>=32B2025.10 | 18.65 | 34.2 | 12.46 | 22.87 | 9.61 | 15.95 | 9.76 | 30.34 | 24.36 | 48.07 | 38.34 | 37.29 | 20.37 | |
| HuatuoGPT-o1-8BModel Category=Open-Source Models, Scale=7B-8B2025.10 | 16.25 | 34.15 | 18.03 | 17.73 | 8.14 | 15.12 | 10.91 | 26.54 | 21.11 | 54.24 | 41.57 | 33.81 | 17.05 | |
| DoctorAgent-RLModel Category=Open-Source Models, Scale=7B-8B2025.10 | 15.77 | 33.48 | 14.97 | 17.58 | 7.69 | 15.29 | 8.22 | 30.01 | 20.3 | 50.05 | 38.54 | 31.73 | 18.55 | |
| Med42-v2-8BModel Category=Open-Source Models, Scale=7B-8B2025.10 | 14.97 | 28.4 | 18.59 | 14.08 | 7.43 | 14.28 | 8.86 | 32.48 | 20.67 | 45.68 | 42.45 | 31.09 | 17.79 | |
| OpenBioLLM-8BModel Category=Open-Source Models, Scale=7B-8B2025.10 | 8.17 | 18.3 | 6.4 | 6.34 | 4.81 | 7.83 | 2.5 | 16.46 | 12.96 | 25.72 | 24.3 | 23.77 | 10.67 | |
| BioMistral-7BModel Category=Open-Source Models, Scale=7B-8B2025.10 | 6.25 | 12.5 | 3.02 | 5.62 | 3.14 | 7.1 | 3.28 | 11.94 | 11.8 | 19.65 | 23.94 | 23.13 | 8.63 | |
| HippoMistral-7BModel Category=Open-Source Models, Scale=7B-8B2025.10 | 5.93 | 10.04 | 2.07 | 6.12 | 3.32 | 6.08 | 3.02 | 13.75 | 11.54 | 23.37 | 17.83 | 22.4 | 9.31 | |
| HippoLlama-7BModel Category=Open-Source Models, Scale=7B-8B2025.10 | 2.31 | 0.48 | 0.47 | 1.64 | 0.85 | 3.98 | 0.58 | 12.59 | 7.08 | 9.47 | 20.66 | 15.63 | 5.19 |