Chemical Reasoning on SuperChem 500 (Total)
72.8Accuracy (Text)Claude Opus 4.6
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Claude Opus 4.6Type=Commercial LLMs, Setting=BioMedArena2026.05 | 72.8 | 57.9 | 65.8 | |
| GPT-5.4Type=Commercial LLMs, Setting=BioMedArena2026.05 | 66.8 | 58.3 | 62.8 | |
| Claude Sonnet 4.6Type=Commercial LLMs, Setting=BioMedArena2026.05 | 66 | 53.6 | 58.6 | |
| Claude Opus 4.6Type=Commercial LLMs, Setting=Baseline2026.05 | 62.3 | 30.2 | 47.2 | |
| Claude Opus 4.5Type=Commercial LLMs, Setting=BioMedArena2026.05 | 59.2 | 41.3 | 50.8 | |
| GPT-5.4Type=Commercial LLMs, Setting=Baseline2026.05 | 57.7 | 22.6 | 41.2 | |
| SOTASource=[23], [25]2026.05 | 57.5 | 63.2 | — | |
| Gemini 3.1 ProType=Commercial LLMs, Setting=BioMedArena2026.05 | 57.4 | 62.6 | 59.8 | |
| Gemini 3.1 ProType=Commercial LLMs, Setting=Baseline2026.05 | 54.3 | 19.2 | 37.8 | |
| Gemini 3 FlashType=Commercial LLMs, Setting=BioMedArena2026.05 | 54 | 34.5 | 44.8 | |
| Gemini 3 FlashType=Commercial LLMs, Setting=Baseline2026.05 | 49.1 | 10.2 | 30.8 | |
| Claude Opus 4.5Type=Commercial LLMs, Setting=Baseline2026.05 | 48.3 | 24.3 | 37 | |
| Claude Sonnet 4.6Type=Commercial LLMs, Setting=Baseline2026.05 | 40.4 | 24.7 | 33 | |
| Claude Sonnet 4.5Type=Commercial LLMs, Setting=BioMedArena2026.05 | 39.6 | 31.9 | 35.4 | |
| Claude Sonnet 4.5Type=Commercial LLMs, Setting=Baseline2026.05 | 29.1 | 17.4 | 23.6 |