Chemistry Diagram-Grounded Reasoning on OCRD-Bench
92RecGemini 2.5 Pro
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Gemini 2.5 ProFramework=ChemVA, Model Category=Proprietary SOTA Models2026.05 | 92 | 74 | 71 | 76.4 | 17.6 | |
| GPT-5 (Preview)Framework=ChemVA, Model Category=Proprietary SOTA Models2026.05 | 92 | 69 | 66.2 | 72.48 | 18.28 | |
| Qwen3-MaxFramework=ChemVA, Model Category=Proprietary SOTA Models2026.05 | 92 | 70 | 65.7 | 72.68 | 21.28 | |
| GPT-4oFramework=ChemVA, Model Category=Proprietary SOTA Models2026.05 | 92 | 67 | 62.5 | 70.2 | 23.4 | |
| Claude 4 SonnetFramework=ChemVA, Model Category=Proprietary SOTA Models2026.05 | 92 | 66 | 62.5 | 69.8 | 23.12 | |
| Yi-LightningFramework=ChemVA, Model Category=Open-Weights2026.05 | 92 | 58 | 54.7 | 63.48 | 22.2 | |
| Llama-4-109BFramework=ChemVA, Model Category=Open-Weights2026.05 | 92 | 55 | 52 | 61.2 | 22.32 | |
| Intern-S1Framework=ChemVA, Model Category=Open-Weights2026.05 | 92 | 56.5 | 53.5 | 62.4 | 17.72 | |
| ChemVLMFramework=ChemVA, Model Category=Open-Weights2026.05 | 92 | 32.5 | 30.5 | 43.6 | 15.12 | |
| Gemini 2.5 ProFramework=Vanilla LLM, Model Category=Proprietary SOTA Models2026.05 | 68 | 59 | 54 | 58.8 | — | |
| GPT-5 (Preview)Framework=Vanilla LLM, Model Category=Proprietary SOTA Models2026.05 | 64 | 53 | 50.5 | 54.2 | — | |
| Qwen3-MaxFramework=Vanilla LLM, Model Category=Proprietary SOTA Models2026.05 | 61 | 50 | 48 | 51.4 | — | |
| Intern-S1Framework=Vanilla LLM, Model Category=Open-Weights2026.05 | 54 | 43.5 | 41.2 | 44.68 | — | |
| Claude 4 SonnetFramework=Vanilla LLM, Model Category=Proprietary SOTA Models2026.05 | 49 | 47 | 45.2 | 46.68 | — | |
| Yi-LightningFramework=Vanilla LLM, Model Category=Open-Weights2026.05 | 47 | 41 | 38.7 | 41.28 | — | |
| GPT-4oFramework=Vanilla LLM, Model Category=Proprietary SOTA Models2026.05 | 45 | 48 | 46.5 | 46.8 | — | |
| Llama-4-109BFramework=Vanilla LLM, Model Category=Open-Weights2026.05 | 44 | 38.5 | 36.7 | 38.88 | — | |
| ChemVLMFramework=Vanilla LLM, Model Category=Open-Weights2026.05 | 36 | 27.5 | 25.7 | 28.48 | — |