Medical Reasoning on HealthBench
70.41AccuracyCOTCAgent
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| COTCAgentBackbone=Baichuan-M22026.05 | 70.41 | — | 68.7 | |
| KAREBackbone=Baichuan-M22026.05 | 69.58 | — | 66.8 | |
| COTCAgentBackbone=Gemini 2.5 Pro2026.05 | 68.81 | — | 67.8 | |
| COTCAgentBackbone=Claude 3.7 Sonnet2026.05 | 68.59 | — | 67.4 | |
| COTCAgentBackbone=GPT-4o2026.05 | 68.43 | — | 66.8 | |
| API toolsBackbone=Baichuan-M22026.05 | 68.19 | — | 65.8 | |
| API toolsBackbone=Gemini 2.5 Pro2026.05 | 63.87 | — | 62.7 | |
| API toolsBackbone=Claude 3.7 Sonnet2026.05 | 63.76 | — | 62.5 | |
| API toolsBackbone=GPT-4o2026.05 | 63.41 | — | 61.8 | |
| Google'sBackbone=Baichuan-M22026.05 | 62.87 | — | 60.6 | |
| KAREBackbone=Gemini 2.5 Pro2026.05 | 62.31 | — | 61.2 | |
| KAREBackbone=Claude 3.7 Sonnet2026.05 | 62.18 | — | 60.9 | |
| KAREBackbone=GPT-4o2026.05 | 61.94 | — | 60.7 | |
| DirPredBackbone=Baichuan-M22026.05 | 61.76 | — | 59.7 | |
| Google'sBackbone=Gemini 2.5 Pro2026.05 | 61.44 | — | 60.2 | |
| Google'sBackbone=Claude 3.7 Sonnet2026.05 | 61.22 | — | 60 | |
| Google'sBackbone=GPT-4o2026.05 | 61.08 | — | 59.8 | |
| DirPredBackbone=Gemini 2.5 Pro2026.05 | 60.71 | — | 59.2 | |
| TimeCAPBackbone=Baichuan-M22026.05 | 60.58 | — | 58.6 | |
| DirPredBackbone=Claude 3.7 Sonnet2026.05 | 60.54 | — | 59 | |
| DirPredBackbone=GPT-4o2026.05 | 60.42 | — | 58.7 | |
| TimeCAPBackbone=Gemini 2.5 Pro2026.05 | 59.71 | — | 58.3 | |
| TimeCAPBackbone=Claude 3.7 Sonnet2026.05 | 59.52 | — | 58.1 | |
| TimeCAPBackbone=GPT-4o2026.05 | 59.38 | — | 57.9 | |
| COTCAgentBackbone=Qwen3-Next-80B-A3B2026.05 | 58.63 | — | 56.6 | |
| KAREBackbone=Qwen3-Next-80B-A3B2026.05 | 56.07 | — | 54.1 | |
| API toolsBackbone=Qwen3-Next-80B-A3B2026.05 | 55.74 | — | 53.4 | |
| Google'sBackbone=Qwen3-Next-80B-A3B2026.05 | 55.41 | — | 53.6 | |
| DirPredBackbone=Qwen3-Next-80B-A3B2026.05 | 54.88 | — | 52.8 | |
| TimeCAPBackbone=Qwen3-Next-80B-A3B2026.05 | 53.96 | — | 51.7 | |
| AMARISRubric Strategy=per-instance rubric2026.05 | 34 | — | — | |
| AMARISRubric Strategy=global rubric2026.05 | 33.6 | — | — | |
| RubricHub (RuRL)RL Training Approach=RuRL2026.05 | 33 | — | — | |
| RuscaRLRL Training Approach=RuscaRL2026.05 | 32.9 | — | — | |
| RaRRL Training Approach=RaR2026.05 | 31.2 | — | — | |
| Naive (no RL)RL Training Approach=None2026.05 | 22.7 | — | — | |
| Baichuan-M2-32BParameters=32B2026.02 | — | 60.1 | — | |
| Baichuan-M3Fact-Aware RL=false2026.02 | — | 66.2 | — | |
| Baichuan-M3-235BFact-Aware RL=true, Parameters=235B2026.02 | — | 65.1 | — | |
| GPT-5.2-High2026.02 | — | 63.3 | — |