Medical LLM Alignment on ProMedical-Bench
91.5Pointwise Proficiency (S1)GPT-5
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GPT-5Model Category=Closed-Source Generative Models, Access Type=Closed-Source2026.04 | 91.5 | 90.88 | 76.45 | 92.06 | 91.94 | 77.39 | 76.42 | |
| ProMedical-RM-8B (Qwen3)Model Category=Open-Source Reward Models, Access Type=Open-Source, Medical-specific optimization=true, Backbone=Qwen32026.04 | 90.85 | 92.8 | 88.5 | 90.26 | 92.06 | 87.39 | 86.55 | |
| ProMedical-RM-8B (Llama)Model Category=Open-Source Reward Models, Access Type=Open-Source, Medical-specific optimization=true, Backbone=Llama-3-8B-Instruct2026.04 | 90.15 | 91.9 | 87.2 | 89.65 | 91.25 | 86.1 | 85.4 | |
| Gemini-3-ProModel Category=Closed-Source Generative Models, Access Type=Closed-Source2026.04 | 89.8 | 91.2 | 64.1 | 91.2 | 92.06 | 65.65 | 64.8 | |
| DeepSeek-R1Model Category=Open-Source Generative Models, Access Type=Open-Source2026.04 | 89.5 | 88.1 | 78.8 | 90.84 | 89.09 | 80 | 78.55 | |
| Qwen3-235B-ThinkingModel Category=Open-Source Generative Models, Access Type=Open-Source, Parameter Scale=235B2026.04 | 88.4 | 87.9 | 78.1 | 89.1 | 88.5 | 79.2 | 77.45 | |
| PairRM-LLaMA3-8BModel Category=Open-Source Reward Models, Access Type=Open-Source, Backbone=LLaMA3-8B2026.04 | 76.5 | 79.1 | 58.8 | 79.39 | 81.7 | 60.43 | 58.95 | |
| medical_o1_verifier_3BModel Category=Open-Source Reward Models, Access Type=Open-Source, Medical-specific optimization=true, Parameter Scale=3B2026.04 | 75.2 | 71.5 | 51.9 | 77.16 | 73.33 | 53.04 | 51.1 | |
| HuatuoGPT-o1Model Category=Open-Source Generative Models, Access Type=Open-Source, Medical-specific optimization=true2026.04 | 65.1 | 62.4 | 58.2 | 66.37 | 63.21 | 59.57 | 55.4 | |
| Meditron-70BModel Category=Open-Source Generative Models, Access Type=Open-Source, Medical-specific optimization=true, Parameter Scale=70B2026.04 | 64.2 | 59.8 | 56.5 | 64.88 | 60.15 | 57.2 | 53.4 | |
| Qwen3-8BModel Category=Open-Source Generative Models, Access Type=Open-Source, Parameter Scale=8B2026.04 | 50.15 | 51.8 | 62.79 | 49.74 | 52.24 | 65.64 | 64.3 |