Multi-step Scientific Tool-use on SciAgentBench Main Results 1.0
32.3SR (w/o Tools)GPT-5
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-5Access=Closed-Source2026.02 | 32.3 | 41.3 | 9 | 0.24 | 46.3 | 43.8 | 28.6 | 32.3 | 60.6 | 38.6 | 30.9 | |
| Grok-4-1Access=Closed-Source2026.02 | 30.4 | 40.3 | 9.9 | 0.25 | 47.2 | 38.2 | 32.4 | 30 | 52.8 | 43.2 | 27.6 | |
| Gemini-2.5-Pro-ThinkAccess=Closed-Source, Reasoning_Mode=Thinking2026.02 | 28.9 | 28.8 | -0.1 | 0.19 | 33.3 | 28.9 | 21.2 | 21.9 | 51.7 | 28.9 | 11.7 | |
| Gemini-2.5-FlashAccess=Closed-Source2026.02 | 28.5 | 32.7 | 4.2 | 0.21 | 38.3 | 32.4 | 28.6 | 17.2 | 49.5 | 32.7 | 22 | |
| O4-miniAccess=Closed-Source2026.02 | 27.8 | 31.1 | 3.3 | 0.24 | 31.2 | 35.5 | 30.6 | 20 | 53.3 | 35.2 | 9.7 | |
| O3Access=Closed-Source2026.02 | 26.6 | 32 | 5.4 | 0.26 | 35.5 | 37.3 | 32.4 | 6.5 | 50.4 | 31.1 | 20.2 | |
| GLM-4.6VAccess=Open-Source, Parameters=>30B2026.02 | 26 | 30.9 | 4.9 | 0.25 | 30.9 | 37.5 | 22.2 | 18.8 | 44.4 | 31.4 | 21.6 | |
| Gemini-2.5-ProAccess=Closed-Source2026.02 | 24.8 | 32.6 | 7.8 | 0.21 | 37.3 | 35.1 | 26.5 | 18.8 | 56.5 | 33.4 | 16.1 | |
| Qwen3-VL-235B-ThinkAccess=Open-Source, Parameters=235B, Reasoning_Mode=Thinking2026.02 | 24.4 | 28 | 3.6 | 0.16 | 30.6 | 29.5 | 22.9 | 22.6 | 53.9 | 26.9 | 15.1 | |
| Qwen3-VL-32B-ThinkAccess=Open-Source, Parameters=32B, Reasoning_Mode=Thinking2026.02 | 24.4 | 27.9 | 3.5 | 0.17 | 33 | 31.2 | 8.8 | 22.6 | 49.1 | 26.2 | 14.6 | |
| SciAgent-8BAccess=Open-Source, Backbone=Qwen3-VL-8B-Inst, Fine-tuned=SFT2026.02 | 23.3 | 30.1 | 6.8 | 0.16 | 33 | 35.2 | 9.1 | 31 | 41.9 | 27.5 | 19.6 | |
| Qwen3-VL-235B-InstAccess=Open-Source, Parameters=235B2026.02 | 23 | 23.9 | 0.9 | 0.16 | 28.1 | 26.5 | 5 | 17.2 | 50 | 21.8 | 2.4 | |
| Qwen3-VL-32B-InstAccess=Open-Source, Parameters=32B2026.02 | 22.8 | 27.4 | 4.6 | 0.15 | 31.8 | 29.3 | 20 | 16.1 | 47.1 | 25.4 | 15.1 | |
| Claude-4-SonnetAccess=Closed-Source2026.02 | 22.4 | 35.9 | 13.5 | 0.19 | 39.4 | 39.5 | 27 | 25 | 55.6 | 36.4 | 16.9 | |
| Qwen3-VL-8B-InstAccess=Open-Source, Parameters=8B2026.02 | 18.4 | 23.4 | 5 | 0.09 | 24 | 28.6 | 7.1 | 24.1 | 38.7 | 20.9 | 8.2 | |
| SciAgent-4BAccess=Open-Source, Backbone=Qwen3-VL-4B-Inst, Fine-tuned=SFT2026.02 | 17.4 | 25.2 | 7.8 | 0.13 | 28.4 | 28.4 | 14.7 | 19.4 | 43.3 | 27.5 | 13.5 | |
| GPT-4oAccess=Closed-Source2026.02 | 17.1 | 18.7 | 1.6 | 0.14 | 21.3 | 20.5 | 8.6 | 16 | 27.3 | 16.4 | 7.5 | |
| Qwen3-VL-4B-InstAccess=Open-Source, Parameters=4B2026.02 | 17 | 19.7 | 2.7 | 0.1 | 23.8 | 20.6 | 10.3 | 13.3 | 40.7 | 15.2 | 10 | |
| Pixtral-12BAccess=Open-Source, Parameters=12B2026.02 | 7.8 | 7.2 | -0.6 | 0.07 | 7.5 | 6.3 | 5.9 | 10 | 12.1 | 2.9 | 3.7 |