Scientific Problem Solving on SciBench-107
62.5Atkins ScoreClaude-Opus-4.5
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Claude-Opus-4.5Framework=Generative Models2026.02 | 62.5 | 37.5 | 62.5 | 62.5 | 56.25 | 37.5 | 42.86 | 90 | 53.85 | 62.5 | 30 | 80 | 58.33 | 56.07 | |
| Gemini-3-ProFramework=Generative Models2026.02 | 37.5 | 37.5 | 25 | 37.5 | 34.38 | 62.5 | 14.29 | 40 | 30.77 | 68.75 | 70 | 90 | 75 | 46.73 | |
| Gemini-3-ProFramework=CodeAct2026.02 | 37.5 | 37.5 | 50 | 25 | 37.5 | 12.5 | 28.57 | 50 | 30.77 | 75 | 90 | 70 | 77.78 | 48.6 | |
| Qwen3-235B-InstructFramework=CodeAct2026.02 | 37.5 | 50 | 50 | 25 | 40.62 | 25 | 38.1 | 50 | 38.46 | 81.25 | 90 | 90 | 86.11 | 55.14 | |
| Qwen3-235B-InstructFramework=EmbodiedAct2026.02 | 37.5 | 37.5 | 62.5 | 25 | 40.62 | 0 | 33.33 | 55.56 | 32.19 | 75 | 90 | 100 | 86.11 | 52.34 | |
| Gemini-3-ProFramework=EmbodiedAct2026.02 | 37.5 | 25 | 50 | 25 | 34.38 | 12.5 | 52.38 | 50 | 43.59 | 93.75 | 80 | 90 | 88.89 | 56.07 | |
| GPT-5-miniFramework=Generative Models2026.02 | 25 | 50 | 50 | 25 | 37.5 | 37.5 | 19.05 | 50 | 30.77 | 31.25 | 70 | 80 | 55.56 | 41.12 | |
| Qwen3-8B-VL-InstructFramework=CodeAct2026.02 | 25 | 12.5 | 0 | 12.5 | 12.5 | 0 | 14.29 | 44.44 | 19.09 | 50 | 80 | 80 | 66.67 | 32.71 | |
| Gemini-3-FlashFramework=CodeAct2026.02 | 25 | 37.5 | 0 | 12.5 | 18.75 | 12.5 | 14.29 | 20 | 15.39 | 50 | 80 | 90 | 69.44 | 34.58 | |
| GPT-5-miniFramework=CodeAct2026.02 | 25 | 25 | 37.5 | 0 | 21.88 | 14.29 | 38.1 | 55.56 | 37.69 | 81.25 | 90 | 90 | 86.11 | 48.6 | |
| GPT-5.2Framework=CodeAct2026.02 | 25 | 25 | 37.5 | 25 | 28.12 | 25 | 38.1 | 50 | 38.46 | 75 | 90 | 80 | 80.56 | 49.53 | |
| Claude-Sonnet-4.5Framework=EmbodiedAct2026.02 | 25 | 25 | 62.5 | 37.5 | 37.5 | 12.5 | 38.1 | 50 | 35.9 | 75 | 80 | 100 | 83.33 | 52.34 | |
| GPT-5.2Framework=EmbodiedAct2026.02 | 25 | 25 | 50 | 25 | 31.25 | 12.5 | 47.62 | 50 | 41.03 | 87.5 | 90 | 90 | 88.89 | 54.21 | |
| Claude-Opus-4.5Framework=EmbodiedAct2026.02 | 25 | 50 | 62.5 | 37.5 | 43.75 | 12.5 | 52.38 | 60 | 46.15 | 87.5 | 90 | 100 | 91.67 | 60.75 | |
| Qwen3-8B-VL-InstructFramework=Generative Models2026.02 | 12.5 | 25 | 12.5 | 0 | 12.5 | 12.5 | 9.52 | 10 | 10.25 | 6.25 | 10 | 20 | 11.11 | 11.21 | |
| Gemini-3-FlashFramework=Generative Models2026.02 | 12.5 | 25 | 25 | 12.5 | 18.75 | 12.5 | 19.05 | 20 | 17.95 | 25 | 40 | 70 | 41.67 | 26.17 | |
| Qwen3-235B-InstructFramework=Generative Models2026.02 | 12.5 | 62.5 | 25 | 0 | 25 | 25 | 14.29 | 30 | 20.52 | 18.75 | 60 | 50 | 38.89 | 28.04 | |
| Claude-Sonnet-4.5Framework=Generative Models2026.02 | 12.5 | 50 | 62.5 | 25 | 37.5 | 50 | 23.81 | 40 | 33.33 | 31.25 | 90 | 90 | 63.89 | 44.86 | |
| Claude-Sonnet-4.5Framework=CodeAct2026.02 | 12.5 | 37.5 | 50 | 25 | 31.25 | 25 | 23.81 | 50 | 30.77 | 37.5 | 80 | 80 | 61.11 | 41.12 | |
| Claude-Opus-4.5Framework=CodeAct2026.02 | 12.5 | 37.5 | 62.5 | 25 | 34.38 | 42.86 | 33.33 | 55.56 | 40.98 | 37.5 | 80 | 80 | 61.11 | 44.86 | |
| Qwen3-8B-VL-InstructFramework=EmbodiedAct2026.02 | 12.5 | 0 | 0 | 25 | 9.38 | 14.29 | 14.29 | 33.33 | 19.17 | 31.25 | 80 | 50 | 50 | 26.17 | |
| Gemini-3-FlashFramework=EmbodiedAct2026.02 | 12.5 | 25 | 50 | 37.5 | 31.25 | 12.5 | 33.33 | 40 | 30.77 | 62.5 | 90 | 90 | 77.78 | 46.73 | |
| GPT-5-miniFramework=EmbodiedAct2026.02 | 12.5 | 37.5 | 37.5 | 25 | 28.12 | 14.29 | 42.86 | 55.56 | 40.26 | 68.75 | 90 | 90 | 80.56 | 49.53 | |
| GPT-5.2Framework=Generative Models2026.02 | 0 | 37.5 | 62.5 | 37.5 | 34.38 | 37.5 | 23.81 | 20 | 25.64 | 75 | 70 | 80 | 75 | 44.86 |