# General AI Assistant tasks on GAIA (Pass@1, Pass@3) Source: [General AI Assistant tasks on GAIA (Pass@1, Pass@3)]() Collection index: [llms.txt]() Data updated: 2026-05-19T10:30:34.968Z Evaluation of general AI assistant capabilities on the GAIA dataset, measured using Pass@1 and Pass@3 scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | General AI Assistant tasks | | Dataset (including any stated split) | GAIA | | Metric | Pass@1 Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 38 of 38 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Pass@1 Score](): Higher is better; unit: PERCENT. - [Pass@3 Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Pass@1 Score | Result date | Report paper | | --- | --- | --- | --- | | AgentOrchestra | 83.4 | Not recorded | 69fc1c945497b3b670c88565 | | Uno-Orchestra | 82 | Not recorded | 69fc1c945497b3b670c88565 | | Tongyi DeepResearch | 70.9 | Not recorded | 69019f853ef7ccd74e5e3b7f | | OpenAI-o3 | 70.5 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | AOrchestra | 69.4 | Not recorded | 69fc1c945497b3b670c88565 | | Claude-4 | 68.3 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | Claude-4-Sonnet | 68.3 | Not recorded | 69019f853ef7ccd74e5e3b7f | | OpenAI DeepResearch | 67.4 | Not recorded | 69019f853ef7ccd74e5e3b7f | | GLM 4.5 | 66 | Not recorded | 69019f853ef7ccd74e5e3b7f | | DeepSeek-v3.1 | 63.1 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | DeepSeek-V3.1 | 63.1 | Not recorded | 69019f853ef7ccd74e5e3b7f | | OpenAI o4-mini | 60 | Not recorded | 69019f853ef7ccd74e5e3b7f | | Kimi-K2 | 57.7 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | Kimi K2 | 57.7 | Not recorded | 69019f853ef7ccd74e5e3b7f | | WebSailor-30B | 51.5 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-30B | 49.2 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-30B | 47.3 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-30B | 46.9 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-30B | 45.6 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-30B | 45 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-7B | 43.4 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-7B | 42.4 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-7B | 41.1 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-3B | 40.5 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-7B | 40.5 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-30B | 40.1 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-3B | 39.2 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-3B | 35.3 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-7B | 34.6 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-30B | 33.3 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-7B | 33 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-3B | 32.4 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-7B | 31.7 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-3B | 27.5 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-3B | 27.2 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | WebSailor-3B | 25.6 | Not recorded | 68fa7c1e1e209131cea8c7b5 | | xRouter | 24.8 | Not recorded | 69fc1c945497b3b670c88565 | | ColdStart-LLM | 17.2 | Not recorded | 69fc1c945497b3b670c88565 | ### AgentOrchestra Pass@1 Score: 83.4. Result ID: 69fc5cd547f34ce924df6e57. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Uno-Orchestra Pass@1 Score: 82. Result ID: 69fc5cd547f34ce924df6e58. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Tongyi DeepResearch Pass@1 Score: 70.9. Result ID: 6a0c3bb2fbc373592c02fbba. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Agent Type":"DeepResearch Agent","Configuration":"30B-A3B"}. This record does not assert independent reproduction. ### OpenAI-o3 Pass@1 Score: 70.5. Result ID: 69c68cf347f34ce924b0c433. [Reporting paper]() Recorded evaluation configuration: {"Paradigm":"ReAct","Summary Tool":"-"}. This record does not assert independent reproduction. ### AOrchestra Pass@1 Score: 69.4. Result ID: 69fc5cd547f34ce924df6e56. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Claude-4 Pass@1 Score: 68.3. Result ID: 69c68cf347f34ce924b0c432. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReAct","Summary Tool":"-"}. This record does not assert independent reproduction. ### Claude-4-Sonnet Pass@1 Score: 68.3. Result ID: 6a0c3bb2fbc373592c02fbb7. [Reporting paper]() Recorded evaluation configuration: {"Agent Type":"LLM-based ReAct Agent"}. This record does not assert independent reproduction. ### OpenAI DeepResearch Pass@1 Score: 67.4. Result ID: 6a0c3bb2fbc373592c02fbb9. [Reporting paper]() Recorded evaluation configuration: {"Agent Type":"DeepResearch Agent"}. This record does not assert independent reproduction. ### GLM 4.5 Pass@1 Score: 66. Result ID: 6a0c3bb2fbc373592c02fbb4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Agent Type":"LLM-based ReAct Agent"}. This record does not assert independent reproduction. ### DeepSeek-v3.1 Pass@1 Score: 63.1. Result ID: 69c68cf347f34ce924b0c435. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReAct","Summary Tool":"-"}. This record does not assert independent reproduction. ### DeepSeek-V3.1 Pass@1 Score: 63.1. Result ID: 6a0c3bb2fbc373592c02fbb6. [Reporting paper]() Recorded evaluation configuration: {"Agent Type":"LLM-based ReAct Agent"}. This record does not assert independent reproduction. ### OpenAI o4-mini Pass@1 Score: 60. Result ID: 6a0c3bb2fbc373592c02fbb8. [Reporting paper]() Recorded evaluation configuration: {"Agent Type":"LLM-based ReAct Agent"}. This record does not assert independent reproduction. ### Kimi-K2 Pass@1 Score: 57.7. Result ID: 69c68cf347f34ce924b0c434. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReAct","Summary Tool":"-"}. This record does not assert independent reproduction. ### Kimi K2 Pass@1 Score: 57.7. Result ID: 6a0c3bb2fbc373592c02fbb5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Agent Type":"LLM-based ReAct Agent"}. This record does not assert independent reproduction. ### WebSailor-30B Pass@1 Score: 51.5. Result ID: 69c68cf347f34ce924b0c449. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReSum","Summary Tool":"GPT-OSS-120B"}. This record does not assert independent reproduction. ### WebSailor-30B Pass@1 Score: 49.2. Result ID: 69c68cf347f34ce924b0c44b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReSum","Summary Tool":"DeepSeek-R1-671B"}. This record does not assert independent reproduction. ### WebSailor-30B Pass@1 Score: 47.3. Result ID: 69c68cf347f34ce924b0c448. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReSum","Summary Tool":"ReSumTool-30B"}. This record does not assert independent reproduction. ### WebSailor-30B Pass@1 Score: 46.9. Result ID: 69c68cf347f34ce924b0c44a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReSum","Summary Tool":"Qwen3-235B"}. This record does not assert independent reproduction. ### WebSailor-30B Pass@1 Score: 45.6. Result ID: 69c68cf347f34ce924b0c447. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReSum","Summary Tool":"Qwen3-30B"}. This record does not assert independent reproduction. ### WebSailor-30B Pass@1 Score: 45. Result ID: 69c68cf347f34ce924b0c444. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReAct","Summary Tool":"-"}. This record does not assert independent reproduction. ### WebSailor-7B Pass@1 Score: 43.4. Result ID: 69c68cf347f34ce924b0c442. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReSum","Summary Tool":"Qwen3-235B"}. This record does not assert independent reproduction. ### WebSailor-7B Pass@1 Score: 42.4. Result ID: 69c68cf347f34ce924b0c441. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReSum","Summary Tool":"GPT-OSS-120B"}. This record does not assert independent reproduction. ### WebSailor-7B Pass@1 Score: 41.1. Result ID: 69c68cf347f34ce924b0c443. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReSum","Summary Tool":"DeepSeek-R1-671B"}. This record does not assert independent reproduction. ### WebSailor-3B Pass@1 Score: 40.5. Result ID: 69c68cf347f34ce924b0c43a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReSum","Summary Tool":"GPT-OSS-120B"}. This record does not assert independent reproduction. ### WebSailor-7B Pass@1 Score: 40.5. Result ID: 69c68cf347f34ce924b0c440. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReSum","Summary Tool":"ReSumTool-30B"}. This record does not assert independent reproduction. ### WebSailor-30B Pass@1 Score: 40.1. Result ID: 69c68cf347f34ce924b0c445. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"Recent History","Summary Tool":"-"}. This record does not assert independent reproduction. ### WebSailor-3B Pass@1 Score: 39.2. Result ID: 69c68cf347f34ce924b0c43c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReSum","Summary Tool":"DeepSeek-R1-671B"}. This record does not assert independent reproduction. ### WebSailor-3B Pass@1 Score: 35.3. Result ID: 69c68cf347f34ce924b0c439. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReSum","Summary Tool":"ReSumTool-30B"}. This record does not assert independent reproduction. ### WebSailor-7B Pass@1 Score: 34.6. Result ID: 69c68cf347f34ce924b0c43f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReSum","Summary Tool":"Qwen3-30B"}. This record does not assert independent reproduction. ### WebSailor-30B Pass@1 Score: 33.3. Result ID: 69c68cf347f34ce924b0c446. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"MEM1","Summary Tool":"-"}. This record does not assert independent reproduction. ### WebSailor-7B Pass@1 Score: 33. Result ID: 69c68cf347f34ce924b0c43e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"Recent History","Summary Tool":"-"}. This record does not assert independent reproduction. ### WebSailor-3B Pass@1 Score: 32.4. Result ID: 69c68cf347f34ce924b0c43b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReSum","Summary Tool":"Qwen3-235B"}. This record does not assert independent reproduction. ### WebSailor-7B Pass@1 Score: 31.7. Result ID: 69c68cf347f34ce924b0c43d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReAct","Summary Tool":"-"}. This record does not assert independent reproduction. ### WebSailor-3B Pass@1 Score: 27.5. Result ID: 69c68cf347f34ce924b0c438. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReSum","Summary Tool":"Qwen3-30B"}. This record does not assert independent reproduction. ### WebSailor-3B Pass@1 Score: 27.2. Result ID: 69c68cf347f34ce924b0c437. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"Recent History","Summary Tool":"-"}. This record does not assert independent reproduction. ### WebSailor-3B Pass@1 Score: 25.6. Result ID: 69c68cf347f34ce924b0c436. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"ReAct","Summary Tool":"-"}. This record does not assert independent reproduction. ### xRouter Pass@1 Score: 24.8. Result ID: 69fc5cd547f34ce924df6e55. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### ColdStart-LLM Pass@1 Score: 17.2. Result ID: 69fc5cd547f34ce924df6e54. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.