Medical Dialogue on MAQuE
67AccuracyGPT-5
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| GPT-5Model Category=Proprietary Models2025.10 | 67 | 79.3 | 29.5 | 91.3 | 96.8 | 81.4 | 71.6 | 47.8 | |
| GPT-4.1Model Category=Proprietary Models2025.10 | 60 | 70.59 | 46.02 | 89.53 | 98 | 86.6 | 77.8 | 75.2 | |
| DOCTOR-R1Ablation Study=Full Model2025.10 | 60 | 77.03 | 38.52 | 87.5 | 70.2 | 76.2 | 69.8 | 93.8 | |
| Grok-4Model Category=Proprietary Models2025.10 | 58 | 74.22 | 31.26 | 89.55 | 90.4 | 81.8 | 80.2 | 90.6 | |
| Gemini-2.5-FlashModel Category=Proprietary Models2025.10 | 57 | 70.74 | 33.57 | 65.84 | 83.8 | 65.8 | 68.6 | 45.8 | |
| Baichuan-M2-32BModel Category=Open-Source Models, Scale=>=32B2025.10 | 57 | 66.41 | 44.59 | 90.78 | 97 | 79 | 72.4 | 65.2 | |
| DOCTOR-R1Ablation Study=w/o Experience2025.10 | 56 | 70.6 | 34.46 | 84.66 | 77.8 | 75.4 | 65.4 | 82.2 | |
| UltraMedical-8BModel Category=Open-Source Models, Scale=8B2025.10 | 52 | 66.75 | 25.14 | 81 | 37 | 61 | 58.2 | 80.8 | |
| UltraMedical-70BModel Category=Open-Source Models, Scale=>=32B2025.10 | 52 | 71.59 | 39.11 | 84.36 | 78.8 | 85.8 | 75 | 83.4 | |
| DOCTOR-R1Ablation Study=w/o Proc. Reward2025.10 | 52 | 66.18 | 36.14 | 82.1 | 53.8 | 75.6 | 65 | 63.2 | |
| DoctorAgent-RLModel Category=Open-Source Models, Scale=8B2025.10 | 50 | 67.15 | 35.39 | 74.9 | 68.4 | 66 | 74.8 | 66.6 | |
| Base (Qwen3-8B)Ablation Study=Base Model2025.10 | 49 | 65.2 | 33.5 | 74.09 | 55.2 | 69.4 | 66 | 43.2 | |
| Med42-v2-8BModel Category=Open-Source Models, Scale=8B2025.10 | 45 | 66.69 | 30.01 | 61.48 | 47.4 | 65 | 57.6 | 64.2 | |
| HuatuoGPT-o1-8BModel Category=Open-Source Models, Scale=8B2025.10 | 40 | 62.62 | 21.35 | 89.38 | 36.2 | 63.6 | 57.2 | 60.8 |