EHR-based clinical information retrieval and question answering on AgentEHR-Bench (test)
58.5Diagnoses ScoreClaude Opus 4.6
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Claude Opus 4.6Model Access=Closed-source2026.05 | 58.5 | 42.1 | 27.2 | 31.1 | 20.9 | 36 | |
| ClinSeek-35B-A3BModel Access=Open-source, Configuration=SFT2026.05 | 55.4 | 38.5 | 27.6 | 31.7 | 16.7 | 34 | |
| Claude Sonnet 4.6Model Access=Closed-source2026.05 | 54.4 | 35.6 | 23.4 | 26.3 | 23.7 | 32.7 | |
| MiniMax-M2.5Model Access=Open-source2026.05 | 51.5 | 29 | 19 | 22 | 17 | 27.7 | |
| Kimi K2.5Model Access=Open-source2026.05 | 46.9 | 33.7 | 18.9 | 27.9 | 22.1 | 29.9 | |
| GLM-4.7Model Access=Open-source2026.05 | 46.4 | 28.6 | 16.6 | 23.7 | 22.9 | 27.6 | |
| Qwen3.5-35B-A3B (base)Model Access=Open-source, Configuration=base2026.05 | 36.6 | 17.7 | 16.2 | 21.9 | 18.1 | 22.1 | |
| Qwen3-235B-A22BModel Access=Open-source2026.05 | 30.6 | 20.3 | 17.3 | 24.9 | 9.6 | 20.5 | |
| gpt-oss-120bModel Access=Open-source2026.05 | 27.3 | 12.8 | 12.4 | 19.1 | 7.6 | 15.8 | |
| Tongyi DeepResearch 30B-A3BModel Access=Open-source2026.05 | 25.8 | 14.9 | 8.8 | 17.9 | 13.2 | 16.1 | |
| OpenSeeker-30BModel Access=Open-source2026.05 | 20.4 | 4.5 | 12.8 | 14.2 | 10.6 | 12.5 | |
| Gemma-4-26B-A4B-itModel Access=Open-source2026.05 | 17.9 | 18.5 | 19.7 | 11.2 | 8.8 | 15.2 |