Medical Agent Task Execution on MedAgentBench
79.3Success RateClaude-4-Sonnet
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| Claude-4-SonnetCategory=Frontier (≫8B), Temperature=1, Top-p=1, Number of runs=42026.03 | 79.3 | — | — | — | |
| Gemini-3-ProCategory=Frontier (≫8B), Temperature=1, Top-p=1, Number of runs=42026.03 | 74.8 | — | — | — | |
| SELSMBase Model=Qwen3-30B-A3B2026.03 | 71.33 | 100 | 69.33 | 73.33 | |
| DeepSeek-V3.2-ExpCategory=Frontier (≫8B), Temperature=1, Top-p=1, Number of runs=42026.03 | 67.3 | — | — | — | |
| Gemini-2.5-ProCategory=Frontier (≫8B), Temperature=1, Top-p=1, Number of runs=42026.03 | 65.1 | — | — | — | |
| GPT-OSS-120BCategory=Frontier (≫8B), Temperature=1, Top-p=1, Number of runs=42026.03 | 64.3 | — | — | — | |
| Kimi-K2-0905Category=Frontier (≫8B), Temperature=1, Top-p=1, Number of runs=42026.03 | 61.2 | — | — | — | |
| SELSMBase Model=Average2026.03 | 60.89 | 90 | 59.11 | 62.66 | |
| SELSMBase Model=Qwen3-32B2026.03 | 60.33 | 94.33 | 66.66 | 54 | |
| ExpeLBase Model=Qwen3-30B-A3B2026.03 | 58 | 99.33 | 54.66 | 61.33 | |
| ExpeLBase Model=Qwen3-32B2026.03 | 58 | 95 | 56.66 | 59.33 | |
| DIVE-8B (RL)Category=Ours, Temperature=1, Top-p=1, Number of runs=42026.03 | 57.3 | — | — | — | |
| EnvScaler-8BCategory=8B Baselines, Temperature=1, Top-p=1, Number of runs=42026.03 | 56.6 | — | — | — | |
| SELSMBase Model=GLM4-32B2026.03 | 51 | 75.66 | 41.33 | 60.66 | |
| DIVE-8B (SFT)Category=Ours, Temperature=1, Top-p=1, Number of runs=42026.03 | 50.2 | — | — | — | |
| A-MemBase Model=Qwen3-30B-A3B2026.03 | 47.66 | 100 | 30.66 | 64.66 | |
| ExpeLBase Model=Average2026.03 | 45.55 | 81.44 | 44.22 | 46.89 | |
| Qwen3-8B (base)Category=Ours, Temperature=1, Top-p=1, Number of runs=42026.03 | 38.4 | — | — | — | |
| A-MemBase Model=Average2026.03 | 30.44 | 92.11 | 32 | 28.88 | |
| A-MemBase Model=Qwen3-32B2026.03 | 24 | 89.33 | 35.33 | 12.66 | |
| ExpeLBase Model=GLM4-32B2026.03 | 20.66 | 50 | 21.33 | 20 | |
| A-MemBase Model=GLM4-32B2026.03 | 19.66 | 87 | 30 | 9.33 | |
| WebExplorer-8BCategory=8B Baselines, Temperature=1, Top-p=1, Number of runs=42026.03 | 17.8 | — | — | — | |
| SWE-Dev-8BCategory=8B Baselines, Temperature=1, Top-p=1, Number of runs=42026.03 | 14.2 | — | — | — |