Medical Question Answering on HealthBench Hard
39.02AccuracyPass
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PassK=82026.04 | 39.02 | — | — | |
| PassK=42026.04 | 30.62 | — | — | |
| AggAgentK=82026.04 | 24.46 | — | — | |
| AggAgentK=42026.04 | 24.27 | — | — | |
| SolAggK=82026.04 | 21.84 | — | — | |
| SolAggK=42026.04 | 21.25 | — | — | |
| PassK=22026.04 | 21.09 | — | — | |
| AggAgentK=22026.04 | 19.25 | — | — | |
| SolAggK=22026.04 | 17.3 | — | — | |
| SummAggK=82026.04 | 16.92 | — | — | |
| SummAggK=42026.04 | 12.37 | — | — | |
| BoNK=42026.04 | 10.58 | — | — | |
| BoNK=22026.04 | 9.74 | — | — | |
| SummAggK=22026.04 | 9.74 | — | — | |
| PassK=12026.04 | 9.67 | — | — | |
| FewToolK=22026.04 | 9.02 | — | — | |
| BoNK=82026.04 | 8.79 | — | — | |
| FewToolK=42026.04 | 8.04 | — | — | |
| FewToolK=82026.04 | 5.34 | — | — | |
| Baichuan-M2-32BType=Specialized LLM2026.02 | — | 34.7 | — | |
| Claude-3.7-SonnetType=Closed-source LLM2026.02 | — | 15 | — | |
| Deepseek-R1Type=Open-source LLM2026.02 | — | 15.1 | — | |
| Gemini 2.5 ProDecoding=Greedy2026.02 | — | — | 16.8 | |
| Gemini 3.0 ProDecoding=Greedy2026.02 | — | — | 17.7 | |
| Gemini-2.5-ProType=Closed-source LLM2026.02 | — | 18.5 | — | |
| GPT-4.1Type=Closed-source LLM2026.02 | — | 16 | — | |
| GPT-5.2 ThinkingMode=Thinking mode, Decoding=Greedy2026.02 | — | — | 42.04 | |
| HuatuoGPT-o1-72BType=Specialized LLM2026.02 | — | 18.2 | — | |
| MedXIAOHEMode=Thinking mode, Decoding=Greedy2026.02 | — | — | 46.1 | |
| o3Type=Closed-source LLM2026.02 | — | 31.6 | — | |
| Qwen3-235B-InstructType=Open-source LLM2026.02 | — | 16.2 | — | |
| Qwen3-30B-A3B-InstructType=Our Method baseline2026.02 | — | 15 | — | |
| Qwen3-30B-A3B-Instruct + More Query Rubricsbackbone=Qwen3-30B-A3B-Instruct, supervision=More Query Rubrics2026.02 | — | 33.4 | — | |
| Qwen3-32BType=Open-source LLM2026.02 | — | 12 | — | |
| Qwen3-4B-InstructType=Our Method baseline2026.02 | — | 5.2 | — | |
| Qwen3-4B-Instruct + Doctor Rubricsbackbone=Qwen3-4B-Instruct, supervision=Doctor Rubrics2026.02 | — | 22.9 | — | |
| Qwen3-4B-Instruct + Draft Rubricsbackbone=Qwen3-4B-Instruct, supervision=Draft Rubrics2026.02 | — | 21.2 | — | |
| Qwen3-4B-Instruct + More Query Rubricsbackbone=Qwen3-4B-Instruct, supervision=More Query Rubrics2026.02 | — | 27.2 | — | |
| Qwen3-4B-Instruct + Principle Rubricsbackbone=Qwen3-4B-Instruct, supervision=Principle Rubrics2026.02 | — | 24.4 | — |