# Spreadsheet Manipulation on SpreadsheetBench Source: [Spreadsheet Manipulation on SpreadsheetBench]() Collection index: [llms.txt]() Data updated: 2026-05-22T09:29:42.316Z Evaluation of model performance on spreadsheet manipulation tasks using the SpreadsheetBench dataset, measured by Pass@1. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Spreadsheet Manipulation | | Dataset (including any stated split) | SpreadsheetBench | | Metric | Pass@1 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Pass@1](): Higher is better; unit: PERCENT. ## Recorded results | Method | Pass@1 | Result date | Report paper | | --- | --- | --- | --- | | Copilot Agent Mode | 57.7 | Not recorded | 6a0fe33ff984739ed6739ff2 | | ChatGPT agent with .xlsx access | 45.5 | Not recorded | 6a0fe33ff984739ed6739ff2 | | Claude Files Opus 4.1 | 42.9 | Not recorded | 6a0fe33ff984739ed6739ff2 | | ChatGPT agent | 35.3 | Not recorded | 6a0fe33ff984739ed6739ff2 | | Qwen3-4B-Thinking-2507 + Spreadsheet-RL post-training | 23.4 | Not recorded | 6a0fe33ff984739ed6739ff2 | | OpenAI o3 | 23.3 | Not recorded | 6a0fe33ff984739ed6739ff2 | | Qwen3-4B-Thinking-2507 + Comprehensive spreadsheet-tool access | 19.3 | Not recorded | 6a0fe33ff984739ed6739ff2 | | GPT-4o | 18.4 | Not recorded | 6a0fe33ff984739ed6739ff2 | | Qwen3-32B | 17.6 | Not recorded | 6a0fe33ff984739ed6739ff2 | | GPT-4o | 16.8 | Not recorded | 6a0fe33ff984739ed6739ff2 | | Qwen3-8B | 15.9 | Not recorded | 6a0fe33ff984739ed6739ff2 | | Qwen3-4B-Thinking-2507 + Spreadsheet-native interaction harness | 15.6 | Not recorded | 6a0fe33ff984739ed6739ff2 | | Qwen3-14B | 15 | Not recorded | 6a0fe33ff984739ed6739ff2 | | Qwen3-4B-Thinking-2507 | 12 | Not recorded | 6a0fe33ff984739ed6739ff2 | | Qwen3-4B | 11 | Not recorded | 6a0fe33ff984739ed6739ff2 | | Qwen3-4B-Instruct-2507 | 9.3 | Not recorded | 6a0fe33ff984739ed6739ff2 | ### Copilot Agent Mode Pass@1: 57.7. Result ID: 6a1021f9fbc373592c0979e2. [Reporting paper]() Recorded evaluation configuration: {"Environment Access":"Windows, Excel"}. This record does not assert independent reproduction. ### ChatGPT agent with .xlsx access Pass@1: 45.5. Result ID: 6a1021f9fbc373592c0979e1. [Reporting paper]() Recorded evaluation configuration: {"Environment Access":"OSX, LibreOffice"}. This record does not assert independent reproduction. ### Claude Files Opus 4.1 Pass@1: 42.9. Result ID: 6a1021f9fbc373592c0979e0. [Reporting paper]() Recorded evaluation configuration: {"Environment Access":"Windows, Excel"}. This record does not assert independent reproduction. ### ChatGPT agent Pass@1: 35.3. Result ID: 6a1021f9fbc373592c0979df. [Reporting paper]() Recorded evaluation configuration: {"Environment Access":"OSX, LibreOffice"}. This record does not assert independent reproduction. ### Qwen3-4B-Thinking-2507 + Spreadsheet-RL post-training Pass@1: 23.4. Result ID: 6a1021f9fbc373592c0979eb. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Environment Access":"Spreadsheet Gym"}. This record does not assert independent reproduction. ### OpenAI o3 Pass@1: 23.3. Result ID: 6a1021f9fbc373592c0979de. [Reporting paper]() Recorded evaluation configuration: {"Environment Access":"OSX, LibreOffice"}. This record does not assert independent reproduction. ### Qwen3-4B-Thinking-2507 + Comprehensive spreadsheet-tool access Pass@1: 19.3. Result ID: 6a1021f9fbc373592c0979ea. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Environment Access":"Spreadsheet Gym"}. This record does not assert independent reproduction. ### GPT-4o Pass@1: 18.4. Result ID: 6a1021f9fbc373592c0979dd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Environment Access":"Windows, Excel"}. This record does not assert independent reproduction. ### Qwen3-32B Pass@1: 17.6. Result ID: 6a1021f9fbc373592c0979e7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Environment Access":"Spreadsheet Gym"}. This record does not assert independent reproduction. ### GPT-4o Pass@1: 16.8. Result ID: 6a1021f9fbc373592c0979dc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Environment Access":"OSX, LibreOffice"}. This record does not assert independent reproduction. ### Qwen3-8B Pass@1: 15.9. Result ID: 6a1021f9fbc373592c0979e6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Environment Access":"Spreadsheet Gym"}. This record does not assert independent reproduction. ### Qwen3-4B-Thinking-2507 + Spreadsheet-native interaction harness Pass@1: 15.6. Result ID: 6a1021f9fbc373592c0979e9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Environment Access":"Spreadsheet Gym"}. This record does not assert independent reproduction. ### Qwen3-14B Pass@1: 15. Result ID: 6a1021f9fbc373592c0979e5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Environment Access":"Spreadsheet Gym"}. This record does not assert independent reproduction. ### Qwen3-4B-Thinking-2507 Pass@1: 12. Result ID: 6a1021f9fbc373592c0979e8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Environment Access":"Spreadsheet Gym"}. This record does not assert independent reproduction. ### Qwen3-4B Pass@1: 11. Result ID: 6a1021f9fbc373592c0979e4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Environment Access":"Spreadsheet Gym"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct-2507 Pass@1: 9.3. Result ID: 6a1021f9fbc373592c0979e3. [Reporting paper]() Recorded evaluation configuration: {"Environment Access":"Spreadsheet Gym"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.