Open-ended Medical Consultation on HealthBench Hard
46.2Total ScoreGPT-5 (thinking)
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-5 (thinking)Model Category=Proprietary Models, Model Type=Thinking2025.10 | 46.2 | — | — | — | — | — | — | — | — | — | — | — | — | |
| InfiMed-ORBIT-4B (28k)Model Category=Open-source Models (< 10B), Training Sample Size=28k2025.10 | 37.3 | 51.6 | 50.5 | 41.9 | 10.8 | 30.9 | 43.8 | 13.4 | 38.1 | 48.9 | 42.1 | 31.3 | 49.1 | |
| Baichuan-M2-32BModel Category=Open-source Models (> 10B)2025.10 | 34.5 | 45.6 | 39.5 | 35.6 | 21.3 | 32 | 40.9 | 19.9 | 41.3 | 44.6 | 51.6 | 19.3 | 48 | |
| InfiMed-ORBIT-4B (8k)Model Category=Open-source Models (< 10B), Training Sample Size=8k2025.10 | 33.6 | 44.6 | 49.1 | 34.6 | 9.3 | 28 | 42.6 | 10.4 | 33.6 | 42 | 46.7 | 32.3 | 50.5 | |
| InfiMed-ORBIT-4B (2k)Model Category=Open-source Models (< 10B), Training Sample Size=2k2025.10 | 27.5 | 39.9 | 37.8 | 30.2 | 6.2 | 26.6 | 32.2 | 6.6 | 31.8 | 38.1 | 45.3 | 16.8 | 43.7 | |
| Qwen3-30B-A3B-ThinkingModel Category=Open-source Models (> 10B), Model Type=Thinking2025.10 | 16.1 | 21.4 | 20.4 | 15 | 8.9 | 16.7 | 20.4 | 6.5 | 33.7 | 11.6 | 53 | 0 | 45.5 | |
| GPT-4.1Model Category=Proprietary Models2025.10 | 13.2 | 20.5 | 12.3 | 12.1 | 9.7 | 14.9 | 12.3 | 17.5 | 30.5 | 0 | 70.6 | 0 | 60.5 | |
| Qwen3-30B-InstructModel Category=Open-source Models (> 10B), Model Type=Instruct2025.10 | 13.1 | 18.3 | 12.9 | 14.7 | 17.9 | 19.4 | 9.5 | 28.5 | 28.5 | 0 | 45.2 | 0 | 33.7 | |
| Qwen3-4B-Instruct (base)Model Category=Open-source Models (< 10B), Model Type=Instruct (base)2025.10 | 7 | 9.3 | 8.5 | 7.1 | 0 | 8.6 | 12.2 | 5.1 | 24.1 | 0.8 | 57.5 | 0 | 45 | |
| Qwen3-4B-ThinkingModel Category=Open-source Models (< 10B), Model Type=Thinking2025.10 | 5.2 | 14.4 | 12.5 | 2.4 | 0 | 3.5 | 8.5 | 0 | 23.2 | 0 | 42.5 | 0 | 39.6 | |
| Qwen-2.5-7B-InstructModel Category=Open-source Models (< 10B), Model Type=Instruct2025.10 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 6.4 | 0 | 45.2 | 0 | 33.7 |