# Multi-step Scientific Tool-use on SciAgentBench Main Results 1.0 Source: [Multi-step Scientific Tool-use on SciAgentBench Main Results 1.0]() Collection index: [llms.txt]() Data updated: 2026-02-26T06:03:10.042Z Evaluation of model performance on multi-step scientific tool-use tasks using the SciAgentBench Main Results 1.0 dataset, measured by Success Rate (SR) with and without tools, SPL, and domain-specific scores (Physics, Chemistry, Materials, Life Sciences). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multi-step Scientific Tool-use | | Dataset (including any stated split) | SciAgentBench Main Results 1.0 | | Metric | SR (w/o Tools) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 19 of 19 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [SR (w/o Tools)](): Higher is better; unit: PERCENT. - [SR (w/ Tools)](): Higher is better; unit: PERCENT. - [Delta (Improvement)](): Higher is better; unit: SCALAR. - [SPL](): Higher is better; unit: SCALAR. - [Physics SR](): Higher is better; unit: PERCENT. - [Chemistry SR](): Higher is better; unit: PERCENT. - [Materials SR](): Higher is better; unit: PERCENT. - [Life Sciences SR](): Higher is better; unit: PERCENT. - [SR (Level 1)](): Higher is better; unit: PERCENT. - [SR (Level 2)](): Higher is better; unit: PERCENT. - [SR (Level 3)](): Higher is better; unit: PERCENT. ## Recorded results | Method | SR (w/o Tools) | Result date | Report paper | | --- | --- | --- | --- | | GPT-5 | 32.3 | Not recorded | 6992a47b9ebc499c97406c24 | | Grok-4-1 | 30.4 | Not recorded | 6992a47b9ebc499c97406c24 | | Gemini-2.5-Pro-Think | 28.9 | Not recorded | 6992a47b9ebc499c97406c24 | | Gemini-2.5-Flash | 28.5 | Not recorded | 6992a47b9ebc499c97406c24 | | O4-mini | 27.8 | Not recorded | 6992a47b9ebc499c97406c24 | | O3 | 26.6 | Not recorded | 6992a47b9ebc499c97406c24 | | GLM-4.6V | 26 | Not recorded | 6992a47b9ebc499c97406c24 | | Gemini-2.5-Pro | 24.8 | Not recorded | 6992a47b9ebc499c97406c24 | | Qwen3-VL-235B-Think | 24.4 | Not recorded | 6992a47b9ebc499c97406c24 | | Qwen3-VL-32B-Think | 24.4 | Not recorded | 6992a47b9ebc499c97406c24 | | SciAgent-8B | 23.3 | Not recorded | 6992a47b9ebc499c97406c24 | | Qwen3-VL-235B-Inst | 23 | Not recorded | 6992a47b9ebc499c97406c24 | | Qwen3-VL-32B-Inst | 22.8 | Not recorded | 6992a47b9ebc499c97406c24 | | Claude-4-Sonnet | 22.4 | Not recorded | 6992a47b9ebc499c97406c24 | | Qwen3-VL-8B-Inst | 18.4 | Not recorded | 6992a47b9ebc499c97406c24 | | SciAgent-4B | 17.4 | Not recorded | 6992a47b9ebc499c97406c24 | | GPT-4o | 17.1 | Not recorded | 6992a47b9ebc499c97406c24 | | Qwen3-VL-4B-Inst | 17 | Not recorded | 6992a47b9ebc499c97406c24 | | Pixtral-12B | 7.8 | Not recorded | 6992a47b9ebc499c97406c24 | ### GPT-5 SR (w/o Tools): 32.3. Result ID: 699a5d135698665ec4e4ee20. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Access":"Closed-Source"}. This record does not assert independent reproduction. ### Grok-4-1 SR (w/o Tools): 30.4. Result ID: 699a5d135698665ec4e4ee21. [Reporting paper]() Recorded evaluation configuration: {"Access":"Closed-Source"}. This record does not assert independent reproduction. ### Gemini-2.5-Pro-Think SR (w/o Tools): 28.9. Result ID: 699a5d135698665ec4e4ee27. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Access":"Closed-Source","Reasoning\_Mode":"Thinking"}. This record does not assert independent reproduction. ### Gemini-2.5-Flash SR (w/o Tools): 28.5. Result ID: 699a5d135698665ec4e4ee23. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Access":"Closed-Source"}. This record does not assert independent reproduction. ### O4-mini SR (w/o Tools): 27.8. Result ID: 699a5d135698665ec4e4ee26. [Reporting paper]() Recorded evaluation configuration: {"Access":"Closed-Source"}. This record does not assert independent reproduction. ### O3 SR (w/o Tools): 26.6. Result ID: 699a5d135698665ec4e4ee25. [Reporting paper]() Recorded evaluation configuration: {"Access":"Closed-Source"}. This record does not assert independent reproduction. ### GLM-4.6V SR (w/o Tools): 26. Result ID: 699a5d135698665ec4e4ee29. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Access":"Open-Source","Parameters":"\>30B"}. This record does not assert independent reproduction. ### Gemini-2.5-Pro SR (w/o Tools): 24.8. Result ID: 699a5d135698665ec4e4ee24. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Access":"Closed-Source"}. This record does not assert independent reproduction. ### Qwen3-VL-235B-Think SR (w/o Tools): 24.4. Result ID: 699a5d135698665ec4e4ee2a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Access":"Open-Source","Parameters":"235B","Reasoning\_Mode":"Thinking"}. This record does not assert independent reproduction. ### Qwen3-VL-32B-Think SR (w/o Tools): 24.4. Result ID: 699a5d135698665ec4e4ee2c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Access":"Open-Source","Parameters":"32B","Reasoning\_Mode":"Thinking"}. This record does not assert independent reproduction. ### SciAgent-8B SR (w/o Tools): 23.3. Result ID: 699a5d135698665ec4e4ee2f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Access":"Open-Source","Backbone":"Qwen3-VL-8B-Inst","Fine-tuned":"SFT"}. This record does not assert independent reproduction. ### Qwen3-VL-235B-Inst SR (w/o Tools): 23. Result ID: 699a5d135698665ec4e4ee2b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Access":"Open-Source","Parameters":"235B"}. This record does not assert independent reproduction. ### Qwen3-VL-32B-Inst SR (w/o Tools): 22.8. Result ID: 699a5d135698665ec4e4ee2d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Access":"Open-Source","Parameters":"32B"}. This record does not assert independent reproduction. ### Claude-4-Sonnet SR (w/o Tools): 22.4. Result ID: 699a5d135698665ec4e4ee22. [Reporting paper]() Recorded evaluation configuration: {"Access":"Closed-Source"}. This record does not assert independent reproduction. ### Qwen3-VL-8B-Inst SR (w/o Tools): 18.4. Result ID: 699a5d135698665ec4e4ee2e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Access":"Open-Source","Parameters":"8B"}. This record does not assert independent reproduction. ### SciAgent-4B SR (w/o Tools): 17.4. Result ID: 699a5d135698665ec4e4ee31. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Access":"Open-Source","Backbone":"Qwen3-VL-4B-Inst","Fine-tuned":"SFT"}. This record does not assert independent reproduction. ### GPT-4o SR (w/o Tools): 17.1. Result ID: 699a5d135698665ec4e4ee28. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Access":"Closed-Source"}. This record does not assert independent reproduction. ### Qwen3-VL-4B-Inst SR (w/o Tools): 17. Result ID: 699a5d135698665ec4e4ee30. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Access":"Open-Source","Parameters":"4B"}. This record does not assert independent reproduction. ### Pixtral-12B SR (w/o Tools): 7.8. Result ID: 699a5d135698665ec4e4ee32. [Reporting paper]() Recorded evaluation configuration: {"Access":"Open-Source","Parameters":"12B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.