General AI Assistant tasks on GAIA (Avg, L1, L2, L3)
80.61Avg PerformanceMemEvolve + Flash-Searcher
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| MemEvolve + Flash-SearcherModel Family=GPT-5-mini, pass@k=32025.12 | 80.61 | 94.34 | 79.07 | 57.69 | — | — | — | — | |
| MemEvolve + Flash-SearcherModel Family=GPT-5-mini, pass@k=22025.12 | 77.58 | 92.45 | 74.42 | 57.69 | — | — | — | — | |
| GPT-5 HighModel Category=Foundation Model2026.05 | 76.4 | — | — | — | — | — | — | — | |
| JoyAgentsModel Family=Claude-4, o4-mini2025.12 | 75.2 | 86.8 | 77.9 | 42.3 | — | — | — | — | |
| Cognitive Kernel-ProModel Family=Claude-3.7 etc., pass@k protocol=pass@32026.02 | 75.15 | 84.91 | 73.26 | 61.54 | — | — | — | — | |
| Cognitive Kernel-ProModel Family=Claude-3.7 etc., pass@k=32025.12 | 75.15 | 84.91 | 73.26 | 61.54 | — | — | — | — | |
| DeepSeek-V3.2Model Category=Foundation Model2026.05 | 75.1 | — | — | — | — | — | — | — | |
| Agent KBModel Family=GPT-4.1, pass@k protocol=pass@32026.02 | 73.94 | 84.91 | 73.26 | 53.85 | — | — | — | — | |
| Agent KBModel Family=GPT-4.1, pass@k=32025.12 | 73.94 | 84.91 | 73.26 | 53.85 | — | — | — | — | |
| MemEvolve + Flash-SearcherModel Family=GPT-5-mini, pass@k=12025.12 | 73.33 | 83.02 | 73.26 | 53.85 | — | — | — | — | |
| TodoEvolve + SmolagentsModel Family=GPT-5-Mini, pass@k protocol=pass@12026.02 | 72.12 | 81.14 | 72.09 | 46.15 | — | — | — | — | |
| MemEvolve + SmolAgentModel Family=GPT-5-mini, pass@k=32025.12 | 72.12 | 88.68 | 68.6 | 50 | — | — | — | — | |
| LangfunModel Family=Claude 3.7 etc.2025.12 | 71.52 | 83.02 | 68.6 | 57.69 | — | — | — | — | |
| TodoEvolve + SmolagentsModel Family=DeepSeek V3.2, pass@k protocol=pass@12026.02 | 70.91 | 84.91 | 67.44 | 53.85 | — | — | — | — | |
| TraseAgentModel Family=Claude etc.2025.12 | 70.3 | 83.02 | 69.77 | 46.15 | — | — | — | — | |
| Flash-SearcherModel Family=GPT-5-mini, pass@k protocol=pass@12026.02 | 69.09 | 79.25 | 69.77 | 46.15 | — | — | — | — | |
| Flash-SearcherModel Family=GPT-5-mini, pass@k=12025.12 | 69.09 | 79.25 | 69.77 | 46.15 | — | — | — | — | |
| MemEvolve + SmolAgentModel Family=GPT-5-mini, pass@k=22025.12 | 67.88 | 84.91 | 63.95 | 46.15 | — | — | — | — | |
| MemEvolve + Flash-SearcherModel Family=DeepSeek V3.2, pass@k=12025.12 | 67.88 | 83.02 | 63.95 | 50 | — | — | — | — | |
| OpenAI Deep ResearchModel Family=o1, o3 etc.2025.12 | 67.36 | 74.29 | 69.06 | 47.6 | — | — | — | — | |
| Agent KBModel Family=GPT-4.1, pass@k protocol=pass@22026.02 | 67.27 | 83.02 | 67.44 | 34.62 | — | — | — | — | |
| Agent KBModel Family=GPT-4.1, pass@k=22025.12 | 67.27 | 83.02 | 67.44 | 34.62 | — | — | — | — | |
| OAgentsModel Family=Claude-3.7 etc., pass@k protocol=pass@12026.02 | 66.67 | 77.36 | 66.28 | 46.15 | — | — | — | — | |
| OAgentsModel Family=Claude-3.7 etc.2025.12 | 66.67 | 77.36 | 66.28 | 46.15 | — | — | — | — | |
| SkillHoneSearch setting=raw open-web setting (no pre-integrated search tools)2026.06 | 64.6 | 76.2 | 66.7 | 31.6 | — | — | — | — | |
| MiniMax-M2.1Model Category=Foundation Model2026.05 | 64.3 | — | — | — | — | — | — | — | |
| MemEvolve + SmolAgentModel Family=GPT-5-mini, pass@k=12025.12 | 64.24 | 83.02 | 58.14 | 46.15 | — | — | — | — | |
| h2oGPTeModel Family=Claude-3.52025.12 | 63.64 | 67.92 | 67.44 | 42.31 | — | — | — | — | |
| Agent KBModel Family=GPT-4.1, pass@k protocol=pass@12026.02 | 61.21 | 79.25 | 58.14 | 34.62 | — | — | — | — | |
| Agent KBModel Family=GPT-4.1, pass@k=12025.12 | 61.21 | 79.25 | 58.14 | 34.62 | — | — | — | — | |
| MemEvolve + Flash-SearcherModel Family=Kimi K2, pass@k=12025.12 | 61.21 | 67.92 | 63.95 | 38.46 | — | — | — | — | |
| OWL WorkforceModel Family=GPT-4o+o3-mini, pass@k protocol=pass@32026.02 | 60.61 | 81.14 | 58.14 | 26.92 | — | — | — | — | |
| Flash-SearcherModel Family=DeepSeek V3.2, pass@k protocol=pass@12026.02 | 60.61 | 79.25 | 53.49 | 46.15 | — | — | — | — | |
| OWL WorkforceModel Family=GPT-4o+o3-mini, pass@k=32025.12 | 60.61 | 81.14 | 58.14 | 26.92 | — | — | — | — | |
| Flash-SearcherModel Family=DeepSeek V3.2, pass@k=12025.12 | 60.61 | 79.25 | 53.49 | 46.15 | — | — | — | — | |
| Cognitive Kernel-ProModel Family=Claude-3.7 etc., pass@k protocol=pass@12026.02 | 60 | 79.25 | 56.98 | 30.77 | — | — | — | — | |
| TodoEvolve + SmolagentsModel Family=Kimi K2, pass@k protocol=pass@12026.02 | 60 | 73.58 | 55.81 | 46.15 | — | — | — | — | |
| Cognitive Kernel-ProModel Family=Claude-3.7 etc., pass@k=12025.12 | 60 | 79.25 | 56.98 | 30.77 | — | — | — | — | |
| OWL RPModel Family=GPT-4o+o3-mini, pass@k protocol=pass@32026.02 | 58.18 | 81.14 | 54.65 | 23.08 | — | — | — | — | |
| OWL RPModel Family=GPT-4o+o3-mini, pass@k=32025.12 | 58.18 | 81.14 | 54.65 | 23.08 | — | — | — | — | |
| DesearchModel Family=GPT-4o2025.12 | 56.97 | 71.7 | 58.14 | 23.08 | — | — | — | — | |
| TapeAgentsModel Family=Claude 3.7 etc., pass@k protocol=pass@12026.02 | 55.76 | 71.7 | 53.49 | 30.77 | — | — | — | — | |
| TapeAgentsModel Family=Claude 3.7 etc.2025.12 | 55.76 | 71.7 | 53.49 | 30.77 | — | — | — | — | |
| SmolagentsModel Family=GPT-5-mini, pass@k protocol=pass@12026.02 | 55.75 | 69.81 | 54.65 | 30.77 | — | — | — | — | |
| SmolagentsModel Family=GPT-5-mini2025.12 | 55.75 | 69.81 | 54.65 | 30.77 | — | — | — | — | |
| AutoAgentModel Family=Claude 3.5 etc., pass@k protocol=pass@12026.02 | 55.15 | 71.7 | 53.4 | 26.92 | — | — | — | — | |
| SmolagentsModel Family=GPT-4.1, pass@k protocol=pass@12026.02 | 55.15 | 67.92 | 53.49 | 34.62 | — | — | — | — | |
| AutoAgentModel Family=Claude 3.5 etc.2025.12 | 55.15 | 71.7 | 53.4 | 26.92 | — | — | — | — | |
| SmolagentsModel Family=GPT-4.12025.12 | 55.15 | 67.92 | 53.49 | 34.62 | — | — | — | — | |
| Flash-SearcherModel Family=Kimi K2, pass@k protocol=pass@12026.02 | 52.12 | 58.49 | 52.33 | 34.62 | — | — | — | — | |
| Flash-SearcherModel Family=Kimi K2, pass@k=12025.12 | 52.12 | 58.49 | 52.33 | 34.62 | — | — | — | — | |
| Hermes-SESearch setting=raw open-web setting (no pre-integrated search tools)2026.06 | 50.4 | 73.8 | 40.9 | 31.6 | — | — | — | — | |
| deep-research agentSearch setting=curated search setting (commercial retrieval services)2026.06 | 48.8 | 61.9 | 47 | 26.3 | — | — | — | — | |
| WebThinker-32B-RLModel Category=Search-Agent-Trained Model2026.05 | 48.5 | 56.4 | 50 | 16.7 | — | — | — | — | |
| Magnetic-1Model Family=OpenAI o1 etc., pass@k protocol=pass@12026.02 | 46.06 | 56.6 | 46.51 | 23.08 | — | — | — | — | |
| Magnetic-1Model Family=OpenAI o1 etc.2025.12 | 46.06 | 56.6 | 46.51 | 23.08 | — | — | — | — | |
| Skill-CreatorSearch setting=raw open-web setting (no pre-integrated search tools)2026.06 | 44.1 | 64.3 | 37.9 | 21.1 | — | — | — | — | |
| Qwen3-8B + ACTGUIDE-RLTraining=ACTGUIDE-RL2026.05 | 41.74 | 51.28 | 36.53 | 33.33 | — | — | — | — | |
| Existing-SkillsSearch setting=raw open-web setting (no pre-integrated search tools)2026.06 | 41.7 | 64.3 | 33.3 | 21.1 | — | — | — | — | |
| ARPO-8BModel Category=Search-Agent-Trained Model2026.05 | 38.8 | 53.9 | 32.7 | 16.7 | — | — | — | — | |
| WebSailor-7BModel Category=Search-Agent-Trained Model2026.05 | 37.9 | — | — | — | — | — | — | — | |
| Qwen3-8B + RLTraining=RL Baseline2026.05 | 36.89 | 46.15 | 32.69 | 25 | — | — | — | — | |
| Qwen3-4B-Instruct + ACTGUIDE-RLTraining=ACTGUIDE-RL2026.05 | 35.92 | 46.15 | 32.69 | 16.66 | — | — | — | — | |
| Qwen3-8BTraining=Base2026.05 | 32.03 | 43.58 | 26.92 | 16.66 | — | — | — | — | |
| DAAO2025.09 | 25.97 | 30.42 | 24 | 8.5 | — | — | — | — | |
| Qwen2.5-7B-Instruct + ACTGUIDE-RLTraining=ACTGUIDE-RL2026.05 | 25.24 | 41.02 | 17.3 | 8.33 | — | — | — | — | |
| Qwen3-4B-Instruct + RLTraining=RL Baseline2026.05 | 25.24 | 33.33 | 25 | 0 | — | — | — | — | |
| Qwen2.5-7B-InstructTraining=Base2026.05 | 22.32 | 35.89 | 15.38 | 8.33 | — | — | — | — | |
| Qwen2.5-3B-Instruct + ACTGUIDE-RLTraining=ACTGUIDE-RL2026.05 | 18.45 | 28.21 | 11.54 | 16.66 | — | — | — | — | |
| MaASBackbone=GPT-4o-mini2025.09 | 17.64 | 20.45 | 18.61 | 6.25 | — | — | — | — | |
| Qwen3-4B-InstructTraining=Base2026.05 | 15.53 | 17.94 | 17.3 | 0 | — | — | — | — | |
| Qwen2.5-3B-Instruct + RLTraining=RL Baseline2026.05 | 11.65 | 15.38 | 7.69 | 16.66 | — | — | — | — | |
| Qwen2.5-7B-Instruct + RLTraining=RL Baseline2026.05 | 11.65 | 20.51 | 7.69 | 0 | — | — | — | — | |
| Qwen2.5-3B-InstructTraining=Base2026.05 | 9.71 | 15.38 | 7.69 | 0 | — | — | — | — | |
| AFlowBackbone=GPT-4o-mini2025.09 | 8 | 10.75 | 8.81 | 4.08 | — | — | — | — | |
| ADAS2025.09 | 6.69 | 13.98 | 4.4 | 0 | — | — | — | — | |
| GPT-4o-mini2025.09 | 4.65 | 7.53 | 4.4 | 0 | — | — | — | — | |
| DeepSeek-R1-Distill-32bModel Scale=32b, Model Family=DeepSeek, Mode=Direct2025.09 | — | — | — | — | 19.2 | 7.8 | 4.1 | 11.7 | |
| Qwen2.5-32b-InstructModel Scale=32b, Model Family=Qwen2.5, Mode=Direct2025.09 | — | — | — | — | 8.8 | 7.7 | 3 | 7.6 | |
| ReActModel Scale=7b, Model Family=Qwen2.52025.09 | — | — | — | — | 6.2 | 3.5 | 1.1 | 4.2 | |
| ReActModel Scale=14b, Model Family=Qwen2.52025.09 | — | — | — | — | 11.4 | 7.1 | 0.9 | 8 | |
| ReAct + GRPOModel Scale=7b, Model Family=Qwen2.52025.09 | — | — | — | — | 17.7 | 14.9 | 4.5 | 14.7 | |
| ReAct + GRPOModel Scale=14b, Model Family=Qwen2.52025.09 | — | — | — | — | 21.6 | 15 | 5.5 | 16.4 | |
| ReAct + Tree-GRPOModel Scale=7b, Model Family=Qwen2.52025.09 | — | — | — | — | 19.3 | 17.5 | 5.7 | 16.8 | |
| ReAct + Tree-GRPOModel Scale=14b, Model Family=Qwen2.52025.09 | — | — | — | — | 20.8 | 24.3 | 7.3 | 21 |