# Multi-step reasoning and tool use on GAIA (val) Source: [Multi-step reasoning and tool use on GAIA (val)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:57:22.956Z Evaluation of model performance on multi-step reasoning and tool use capabilities using the validation split of the GAIA dataset, measured by Pass@1, Pass@3, and Average Pass@1 scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multi-step reasoning and tool use | | Dataset (including any stated split) | GAIA (val) | | Metric | Pass@1 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 12 of 12 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Pass@1](): Higher is better; unit: PERCENT. - [Pass@3](): Higher is better; unit: PERCENT. - [Average Pass@1](): Higher is better; unit: PERCENT. ## Recorded results | Method | Pass@1 | Result date | Report paper | | --- | --- | --- | --- | | AORCHESTRA | 80 | Not recorded | 6982d2fba8b01e76cc8e9a12 | | AORCHESTRA | 67.87 | Not recorded | 6982d2fba8b01e76cc8e9a12 | | OpenHands | 66.06 | Not recorded | 6982d2fba8b01e76cc8e9a12 | | OpenHands | 63.64 | Not recorded | 6982d2fba8b01e76cc8e9a12 | | AORCHESTRA | 60.61 | Not recorded | 6982d2fba8b01e76cc8e9a12 | | Mini-SWE | 58.18 | Not recorded | 6982d2fba8b01e76cc8e9a12 | | OpenHands | 54.55 | Not recorded | 6982d2fba8b01e76cc8e9a12 | | Mini-SWE | 50.3 | Not recorded | 6982d2fba8b01e76cc8e9a12 | | ReAct | 49.09 | Not recorded | 6982d2fba8b01e76cc8e9a12 | | ReAct | 47.88 | Not recorded | 6982d2fba8b01e76cc8e9a12 | | ReAct | 46.7 | Not recorded | 6982d2fba8b01e76cc8e9a12 | | Mini-SWE | 40.61 | Not recorded | 6982d2fba8b01e76cc8e9a12 | ### AORCHESTRA Pass@1: 80. Result ID: 699605ce5698665ec43cf104. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Setup":"Gemini-3-Flash"}. This record does not assert independent reproduction. ### AORCHESTRA Pass@1: 67.87. Result ID: 699605ce5698665ec43cf105. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Setup":"DeepSeek-V3.2"}. This record does not assert independent reproduction. ### OpenHands Pass@1: 66.06. Result ID: 699605ce5698665ec43cf0fe. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Setup":"Gemini-3-Flash"}. This record does not assert independent reproduction. ### OpenHands Pass@1: 63.64. Result ID: 699605ce5698665ec43cf0ff. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Setup":"DeepSeek-V3.2"}. This record does not assert independent reproduction. ### AORCHESTRA Pass@1: 60.61. Result ID: 699605ce5698665ec43cf106. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Setup":"Claude-4.5-haiku"}. This record does not assert independent reproduction. ### Mini-SWE Pass@1: 58.18. Result ID: 699605ce5698665ec43cf101. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Setup":"Gemini-3-Flash"}. This record does not assert independent reproduction. ### OpenHands Pass@1: 54.55. Result ID: 699605ce5698665ec43cf100. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Setup":"Claude-4.5-haiku"}. This record does not assert independent reproduction. ### Mini-SWE Pass@1: 50.3. Result ID: 699605ce5698665ec43cf102. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Setup":"DeepSeek-V3.2"}. This record does not assert independent reproduction. ### ReAct Pass@1: 49.09. Result ID: 699605ce5698665ec43cf0fb. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Setup":"Gemini-3-Flash"}. This record does not assert independent reproduction. ### ReAct Pass@1: 47.88. Result ID: 699605ce5698665ec43cf0fd. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Setup":"Claude-4.5-haiku"}. This record does not assert independent reproduction. ### ReAct Pass@1: 46.7. Result ID: 699605ce5698665ec43cf0fc. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Setup":"DeepSeek-V3.2"}. This record does not assert independent reproduction. ### Mini-SWE Pass@1: 40.61. Result ID: 699605ce5698665ec43cf103. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Setup":"Claude-4.5-haiku"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.