Question Answering on Science Q&A Chemistry
83.8AccuracyClaude-Opus-4.6
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Claude-Opus-4.6Type=Proprietary, Evaluation Protocol=zero-shot2026.04 | 83.8 | 0.027 | 0.09 | 0.838 | |
| Qwen3-8B (CaOPD)Type=Ours, Evaluation Protocol=fine-tuned2026.04 | 80.6 | 0.141 | 0.134 | 0.77 | |
| GPT-5.4Type=Proprietary, Evaluation Protocol=zero-shot2026.04 | 72.9 | 0.08 | 0.154 | 0.802 | |
| Claude-Sonnet-4.5Type=Proprietary, Evaluation Protocol=zero-shot2026.04 | 72.4 | 0.155 | 0.201 | 0.874 | |
| Gemini-3.1-ProType=Proprietary, Evaluation Protocol=zero-shot2026.04 | 71.9 | 0.278 | 0.276 | 0.997 | |
| Qwen3.5-397BType=Open-weight, Evaluation Protocol=zero-shot2026.04 | 71.4 | 0.214 | 0.237 | 0.929 | |
| GPT-5.2Type=Proprietary, Evaluation Protocol=zero-shot2026.04 | 63.3 | 0.106 | 0.173 | 0.608 | |
| Gemini-3.1-Flash-LiteType=Proprietary, Evaluation Protocol=zero-shot2026.04 | 62.9 | 0.36 | 0.36 | 0.989 | |
| Gemini-2.5-ProType=Proprietary, Evaluation Protocol=zero-shot2026.04 | 61.9 | 0.365 | 0.364 | 0.984 | |
| Kimi-K2.5Type=Open-weight, Evaluation Protocol=zero-shot2026.04 | 60 | 0.379 | 0.38 | 0.979 | |
| Gemini-2.5-FlashType=Proprietary, Evaluation Protocol=zero-shot2026.04 | 57.6 | 0.424 | 0.424 | 1 | |
| DeepSeek-V3.1Type=Open-weight, Evaluation Protocol=zero-shot2026.04 | 56.7 | 0.433 | 0.433 | 1 | |
| MiniMax-M2.5Type=Open-weight, Evaluation Protocol=zero-shot2026.04 | 47.6 | 0.498 | 0.495 | 0.974 |