High-Level Expert Knowledge Evaluation on HLE Gold 149
80.5Accuracy (Bio)SOTA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SOTASource=[30], [11]2026.05 | 80.5 | 44.2 | 53.2 | |
| Claude Opus 4.6Type=Commercial LLMs, Setting=BioMedArena2026.05 | 55.1 | 59.5 | 56.4 | |
| Gemini 3 FlashType=Commercial LLMs, Setting=BioMedArena2026.05 | 51.4 | 47.6 | 50.3 | |
| Gemini 3.1 ProType=Commercial LLMs, Setting=BioMedArena2026.05 | 49.5 | 50 | 49.7 | |
| Claude Opus 4.5Type=Commercial LLMs, Setting=BioMedArena2026.05 | 49.5 | 50 | 49.7 | |
| GPT-5.4Type=Commercial LLMs, Setting=BioMedArena2026.05 | 48.6 | 54.8 | 50.3 | |
| GPT-5.4Type=Commercial LLMs, Setting=Baseline2026.05 | 43 | 33.3 | 40.3 | |
| Claude Sonnet 4.5Type=Commercial LLMs, Setting=BioMedArena2026.05 | 42.1 | 40.5 | 41.6 | |
| Claude Sonnet 4.6Type=Commercial LLMs, Setting=BioMedArena2026.05 | 42.1 | 50 | 44.3 | |
| Gemini 3.1 ProType=Commercial LLMs, Setting=Baseline2026.05 | 41.1 | 47.6 | 43 | |
| Gemini 3 FlashType=Commercial LLMs, Setting=Baseline2026.05 | 39.3 | 28.6 | 36.2 | |
| Claude Opus 4.6Type=Commercial LLMs, Setting=Baseline2026.05 | 36.4 | 42.9 | 38.3 | |
| Claude Opus 4.5Type=Commercial LLMs, Setting=Baseline2026.05 | 30.8 | 31 | 30.9 | |
| Nemotron-3 Super 120BType=Public LLMs, Setting=BioMedArena2026.05 | 28 | 33.3 | 29.5 | |
| GLM-4.5Type=Public LLMs, Setting=BioMedArena2026.05 | 28 | 28.6 | 28.2 | |
| Qwen3-235BType=Public LLMs, Setting=BioMedArena2026.05 | 28 | 14.3 | 24.2 | |
| INTELLECT-3.1Type=Public LLMs, Setting=BioMedArena2026.05 | 26.2 | 19 | 24.2 | |
| Claude Sonnet 4.6Type=Commercial LLMs, Setting=Baseline2026.05 | 24.3 | 21.4 | 23.5 | |
| Claude Sonnet 4.5Type=Commercial LLMs, Setting=Baseline2026.05 | 22.4 | 16.7 | 20.8 | |
| INTELLECT-3.1Type=Public LLMs, Setting=Baseline2026.05 | 20.6 | 9.5 | 17.4 | |
| Trinity-LargeType=Public LLMs, Setting=BioMedArena2026.05 | 19.6 | 19 | 19.5 | |
| GLM-4.5Type=Public LLMs, Setting=Baseline2026.05 | 16.8 | 2.4 | 12.8 | |
| Trinity-LargeType=Public LLMs, Setting=Baseline2026.05 | 15 | 7.1 | 12.8 | |
| Qwen3-235BType=Public LLMs, Setting=Baseline2026.05 | 15 | 14.3 | 14.8 | |
| Nemotron-3 Super 120BType=Public LLMs, Setting=Baseline2026.05 | 13.1 | 4.8 | 10.7 |