General AI Assistant Task on GAIA (val)
96.23Level 1 Scoreagent-2030
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| agent-20302025.06 | 96.23 | 90.7 | 57.69 | 87.27 | — | — | — | — | — | — | — | — | |
| AGENTORCHESTRAVariant=Evolved2025.06 | 96.23 | 93.02 | 88.46 | 93.33 | — | — | — | — | — | — | — | — | |
| SMITHAgent Framework Category=w. Experience Sharing, Base Model Series=Claude and OpenAI series2025.12 | 94.3 | 80.2 | 61.5 | — | 81.8 | — | — | — | — | — | — | — | |
| AGENTORCHESTRAVariant=Vanilla2025.06 | 92.45 | 88.37 | 88.46 | 89.7 | — | — | — | — | — | — | — | — | |
| MiroFlow + TPGOBackbone=GPT-5, Optimization Iterations=52026.04 | 90.6 | 79.1 | 63.6 | 81.6 | — | — | — | — | — | — | — | 1,765 | |
| DeSearch2025.06 | 90.57 | 72.01 | 38.46 | 72.73 | — | — | — | — | — | — | — | — | |
| AWorldAgent Framework Category=Multi-Agents w. Tool + Memory, Supervised Fine-tuned=true2025.12 | 88.7 | 77.9 | 53.9 | — | 77.6 | — | — | — | — | — | — | — | |
| AWorld2025.06 | 88.68 | 77.91 | 53.85 | 77.58 | — | — | — | — | — | — | — | — | |
| Langfun2025.06 | 88.68 | 80.23 | 57.69 | 79.39 | — | — | — | — | — | — | — | — | |
| Alita2025.06 | 88.68 | 89.53 | 76.92 | 87.27 | — | — | — | — | — | — | — | — | |
| Co-Sight2025.06 | 86.79 | 73.26 | 42.31 | 72.73 | — | — | — | — | — | — | — | — | |
| ManusAgent Framework Category=Multi-Agents w. Tool + Memory, Base Model Series=Claude and OpenAI series2025.12 | 86.5 | 70.1 | 57.7 | — | 73.9 | — | — | — | — | — | — | — | |
| Manus2025.06 | 86.5 | 70.1 | 57.69 | 73.9 | — | — | — | — | — | — | — | — | |
| OWLAgent Framework Category=Multi-Agents w. Tool + Memory, Base Model Series=Claude-series2025.12 | 84.9 | 67.4 | 42.3 | — | 69.7 | — | — | — | — | — | — | — | |
| Langfun Agent v2.1Open/Close Source=Close Source, Main Model=Claude-3.7-sonnet2025.05 | 83.02 | 68.6 | 57.69 | — | — | — | 71.52 | — | — | — | — | — | |
| Trase Agent v0.3Open/Close Source=Close Source, Main Model=o12025.05 | 83.02 | 69.77 | 46.15 | — | — | — | 70.3 | — | — | — | — | — | |
| OAgentsAgent Framework Category=w. Python Interpreter (w.o. Tool Reuse), Base Model Series=Claude-series2025.12 | 83 | 74.4 | 53.9 | — | 66.7 | 73.9 | — | — | — | — | — | — | |
| MiroFlow + TPGOBackbone=GPT-5, Optimization Iterations=32026.04 | 82.1 | 73.1 | 54.5 | 75.7 | — | — | — | — | — | — | — | 2,278 | |
| OWL-RoleplayingOpen/Close Source=Open Source, Main Model=GPT-4o + o3-mini2025.05 | 81.13 | 54.65 | 23.08 | — | — | — | 58.18 | — | — | — | — | — | |
| MiroFlowBackbone=GPT-5, Optimization Iterations=02026.04 | 78.8 | 77.5 | 44.4 | 73.8 | — | — | — | — | — | — | — | 4,014 | |
| AlitaAgent Framework Category=w. Tool Creation, Base Model Series=Claude and OpenAI series2025.12 | 77.4 | 76.7 | 65.4 | — | 75.2 | 87.3 | — | — | — | — | — | — | |
| MementoAgent Framework Category=w. Experience Sharing, Base Model Series=OpenAI models2025.12 | 77.4 | 69.8 | 61.5 | — | 70.9 | 87.9 | — | — | — | — | — | — | |
| MiroFlow + TPGOBackbone=GPT-5, Optimization Iterations=12026.04 | 76.9 | 71.2 | 45.5 | 70.59 | — | — | — | — | — | — | — | 3,680 | |
| SUPERVISORAGENTBase LLM=GPT-4.1, pass@k=3, framework=SMAS2025.10 | 75.47 | 62.79 | 38.46 | — | — | — | 63.03 | 369.52 | 276.84 | 409.05 | 427.72 | — | |
| o3-DR2025.06 | 74.29 | 69.06 | 47.6 | 67.36 | — | — | — | — | — | — | — | — | |
| SUPERVISORAGENTBase LLM=GPT-4.1, pass@k=2, framework=SMAS2025.10 | 73.58 | 56.98 | 34.62 | — | — | — | 58.79 | 389.54 | 270.07 | 420.97 | 529.2 | — | |
| AutoAgentBackbone LLM=Claude-Sonnet-3.52026.02 | 71.7 | 53.49 | 26.92 | 55.15 | — | — | — | — | — | — | — | — | |
| OWLBackbone LLM=GPT-4.12026.02 | 71.7 | 50 | 26.92 | 53.33 | — | — | — | — | — | — | — | — | |
| Smolagents + PreFlectBackbone LLM=GPT-4.12026.02 | 71.7 | 55.81 | 38.46 | 58.18 | — | — | — | — | — | — | — | — | |
| AutoAgentAgent Framework Category=Multi-Agents w. Tool + Memory, Base Model Series=Claude-series2025.12 | 71.7 | 53.4 | 26.9 | — | 55.2 | — | — | — | — | — | — | — | |
| TapeAgentsAgent Framework Category=Multi-Agents w. Tool + Memory, Base Model Series=Claude-series2025.12 | 71.7 | 53.5 | 30.8 | — | 55.8 | — | — | — | — | — | — | — | |
| SmolagentBase LLM=GPT-4.1, pass@k=32025.10 | 71.7 | 63.95 | 34.62 | — | — | — | 61.82 | 502.4 | 282.14 | 605.05 | 611.87 | — | |
| desearchOpen/Close Source=Close Source, Main Model=GPT-4o2025.05 | 71.7 | 58.14 | 23.08 | — | — | — | 56.97 | — | — | — | — | — | |
| TapeAgentsOpen/Close Source=Open Source, Main Model=Claude-3.7-Sonnet2025.05 | 71.7 | 53.49 | 30.77 | — | — | — | 55.76 | — | — | — | — | — | |
| Auto-Deep-ResearchOpen/Close Source=Open Source, Main Model=Claude-3.7-Sonnet2025.05 | 71.7 | 53.49 | 26.92 | — | — | — | 55.15 | — | — | — | — | — | |
| AutoAgentOpen/Close Source=Open Source2025.05 | 71.7 | 53.49 | 30.77 | — | — | — | 55.15 | — | — | — | — | — | |
| Gemini-3-ProNumber of Agents=3, Backbone Model=Gemini-3-Pro2026.01 | 69.81 | 62.79 | 34.62 | 60.61 | — | — | — | — | — | — | — | — | |
| SmolagentBase LLM=GPT-4.1, pass@k=22025.10 | 69.81 | 59.3 | 30.77 | — | — | — | 58.18 | 467.19 | 275.85 | 548.02 | 589.92 | — | |
| Ormind v0.1Open/Close Source=Close Source2025.05 | 69.81 | 54.65 | 26.92 | — | — | — | 55.15 | — | — | — | — | — | |
| WebShaper-32BAgent Framework Category=Multi-Agents w. Tool + Memory, Supervised Fine-tuned=true2025.12 | 69.2 | 50 | 16.6 | — | 53.3 | 61.2 | — | — | — | — | — | — | |
| HF-ODRBackbone LLM=o12026.02 | 67.92 | 53.49 | 34.62 | 55.15 | — | — | — | — | — | — | — | — | |
| OAgentBackbone LLM=GPT-4.12026.02 | 67.92 | 53.49 | 34.62 | 55.15 | — | — | — | — | — | — | — | — | |
| Smolagents + PreFlectBackbone LLM=Gemini-2.5-pro2026.02 | 67.92 | 60.47 | 38.46 | 59.39 | — | — | — | — | — | — | — | — | |
| GPT-5-HighNumber of Agents=3, Backbone Model=GPT-5-High2026.01 | 67.92 | 41.86 | 23.08 | 47.27 | — | — | — | — | — | — | — | — | |
| AWorldBase LLM=GPT-4.12025.10 | 67.92 | 62.79 | 34.62 | — | — | — | 60 | 128.27 | 69.61 | 164.08 | 133.65 | — | |
| h2oGPTe Agent v1.6.8Open/Close Source=Close Source, Main Model=Claude-3.5-Sonnet2025.05 | 67.92 | 67.44 | 42.31 | — | — | — | 63.64 | — | — | — | — | — | |
| Open Deep ResearchOpen/Close Source=Open Source, Main Model=o12025.05 | 67.92 | 53.49 | 34.62 | — | — | — | 55.15 | — | — | — | — | — | |
| HF ODR2025.06 | 67.92 | 53.49 | 34.62 | 55.15 | — | — | — | — | — | — | — | — | |
| OpenDeepResearchAgent Framework Category=Multi-Agents w. Tool + Memory, Base Model Series=OpenAI models2025.12 | 67.9 | 53.5 | 34.6 | — | 55.2 | — | — | — | — | — | — | — | |
| GPT-5.2Number of Agents=3, Backbone Model=GPT-5.22026.01 | 66.04 | 36.05 | 46.15 | 47.27 | — | — | — | — | — | — | — | — | |
| OAgentsBase LLM=GPT-4.12025.10 | 66.04 | 47.67 | 19.23 | — | — | — | 49.09 | 340.5 | 260.27 | 358.63 | 444.11 | — | |
| AngesOpen/Close Source=Close Source2025.05 | 66.04 | 65.12 | 30.77 | — | — | — | 60 | — | — | — | — | — | |
| IdleSpecBackbone=Gemini-2.5-Flash, Use idle?=true2026.05 | 66 | 46.5 | 32.1 | — | — | — | — | — | — | — | — | — | |
| DeepSeek-V3.2Number of Agents=3, Backbone Model=DeepSeek-V3.22026.01 | 64.15 | 45.35 | 46.15 | 51.52 | — | — | — | — | — | — | — | — | |
| Sleep-time ComputeBackbone=Gemini-2.5-Flash, Use idle?=true2026.05 | 62.9 | 44.2 | 20.5 | — | — | — | — | — | — | — | — | — | |
| VanillaBackbone=Gemini-2.5-Flash, Use idle?=false2026.05 | 62.3 | 39.9 | 25.6 | — | — | — | — | — | — | — | — | — | |
| GLM-4.7Number of Agents=3, Backbone Model=GLM-4.72026.01 | 62.26 | 54.65 | 42.31 | 55.15 | — | — | — | — | — | — | — | — | |
| SmolagentBase LLM=GPT-4.1, pass@k=12025.10 | 62.26 | 53.49 | 19.23 | — | — | — | 50.91 | 527.76 | 298.51 | 619.59 | 691.33 | — | |
| SUPERVISORAGENTBase LLM=GPT-4.1, pass@k=1, framework=SMAS2025.10 | 62.26 | 51.16 | 26.92 | — | — | — | 50.91 | 371.12 | 258.28 | 404.96 | 489.22 | — | |
| Barcelona v0.1Open/Close Source=Close Source, Main Model=Claude-3.5-sonnet2025.05 | 62.26 | 50 | 26.92 | — | — | — | 50.3 | — | — | — | — | — | |
| InfantAgent-NextOpen/Close Source=Open Source, Main Model=Claude-3.7-Sonnet2025.05 | 62.26 | 62.79 | 26.92 | — | — | — | 56.97 | — | — | — | — | — | |
| Sequential RevisionBackbone=Gemini-2.5-Flash, Use idle?=false2026.05 | 61.6 | 42.3 | 21.8 | — | — | — | — | — | — | — | — | — | |
| GPT-5Number of Agents=3, Backbone Model=GPT-52026.01 | 60.38 | 54.65 | 42.31 | 54.55 | — | — | — | — | — | — | — | — | |
| Gemini-2.5-ProNumber of Agents=3, Backbone Model=Gemini-2.5-Pro2026.01 | 60.38 | 53.49 | 46.15 | 54.55 | — | — | — | — | — | — | — | — | |
| MonoScaleNumber of Agents=7, Backbone Model=Qwen-3-30B-A3B-Instruct, Memory Usage=true2026.01 | 60.38 | 47.67 | 30.77 | 49.09 | — | — | — | — | — | — | — | — | |
| MonoScaleNumber of Agents=10, Backbone Model=Qwen-3-30B-A3B-Instruct, Memory Usage=true2026.01 | 60.38 | 55.81 | 42.31 | 55.15 | — | — | — | — | — | — | — | — | |
| omneOpen/Close Source=Close Source, Main Model=o1-preview2025.05 | 60.38 | 44.19 | 23.08 | — | — | — | 46.06 | — | — | — | — | — | |
| HF-ODRBackbone LLM=GPT-4.12026.02 | 58.49 | 50 | 34.62 | 50.3 | — | — | — | — | — | — | — | — | |
| Qwen3-235B-InstructNumber of Agents=3, Backbone Model=Qwen3-235B-Instruct2026.01 | 58.49 | 47.67 | 26.92 | 47.88 | — | — | — | — | — | — | — | — | |
| HuggingFace AgentsOpen/Close Source=Open Source, Main Model=gpt-4o2025.05 | 58.49 | 43.02 | 19.23 | — | — | — | 44.24 | — | — | — | — | — | |
| Magnetic-1Backbone LLM=o12026.02 | 56.6 | 46.51 | 23.08 | 46.06 | — | — | — | — | — | — | — | — | |
| CodeAgentBase LLM=GPT-4.12025.10 | 56.6 | 34.88 | 23.08 | — | — | — | 40 | 120.4 | 92.84 | 131.9 | 138.54 | — | |
| DRP-val-v.1.0Open/Close Source=Close Source2025.05 | 56.6 | 48.84 | 15.38 | — | — | — | 46.06 | — | — | — | — | — | |
| Magnetic-1Open/Close Source=Open Source, Main Model=o12025.05 | 56.6 | 46.51 | 23.08 | — | — | — | 46.06 | — | — | — | — | — | |
| OWLBase LLM=GPT-4.12025.10 | 56.56 | 43.02 | 29.16 | — | — | — | 45.4 | 111.07 | 67.72 | 110.36 | 209.34 | — | |
| Multi-Agent Exp v0.1Open/Close Source=Open Source, Main Model=gpt-4-turbo2025.05 | 54.72 | 38.37 | 11.54 | — | — | — | 39.39 | — | — | — | — | — | |
| SmolAgentsAgent Framework Category=w. Python Interpreter (w.o. Tool Reuse), Base Model Series=OpenAI models2025.12 | 54.7 | 53.5 | 26.9 | — | 49.7 | — | — | — | — | — | — | — | |
| HAL AgentBackbone LLM=GPT-4.12026.02 | 52.83 | 55.81 | 23.08 | 49.7 | — | — | — | — | — | — | — | — | |
| MonoScaleNumber of Agents=5, Backbone Model=Qwen-3-30B-A3B-Instruct, Memory Usage=true2026.01 | 52.83 | 45.35 | 42.31 | 47.27 | — | — | — | — | — | — | — | — | |
| qbnlpOpen/Close Source=Close Source, Main Model=o12025.05 | 52.83 | 45.35 | 23.08 | — | — | — | 44.24 | — | — | — | — | — | |
| LRC-HuaweiOpen/Close Source=Close Source2025.05 | 52.83 | 43.02 | 7.69 | — | — | — | 40.61 | — | — | — | — | — | |
| Agent-On-the-flyOpen/Close Source=Close Source, Main Model=QwQ-32B2025.05 | 52.83 | 43.02 | 7.69 | — | — | — | 40.61 | — | — | — | — | — | |
| AgentIM v1.1Open/Close Source=Close Source, Main Model=gpt-4-turbo2025.05 | 50.94 | 40.7 | 15.38 | — | — | — | 40 | — | — | — | — | — | |
| MonoScaleNumber of Agents=3, Backbone Model=Qwen-3-30B-A3B-Instruct, Memory Usage=false2026.01 | 49.06 | 47.67 | 26.92 | 44.84 | — | — | — | — | — | — | — | — | |
| CADMAS-CTXAPI Calls/Task=5.12026.04 | 48.5 | — | 18 | 44.2 | — | — | — | — | — | — | — | — | |
| StaticAPI Calls/Task=4.52026.04 | 48.2 | — | 5.1 | 38.1 | — | — | — | — | — | — | — | — | |
| Learned RouterAPI Calls/Task=3.22026.04 | 47 | — | 7.2 | 36.2 | — | — | — | — | — | — | — | — | |
| AutoGenAPI Calls/Task=18.42026.04 | 46.1 | — | 6.5 | 35.4 | — | — | — | — | — | — | — | — | |
| FRIDAYOpen/Close Source=Open Source, Main Model=gpt-4-turbo2025.05 | 45.28 | 34.88 | 11.54 | — | — | — | 34.55 | — | — | — | — | — | |
| IdleSpecBackbone=Gemma4-E4B, Use idle?=true2026.05 | 43.4 | 28.3 | 14.1 | — | — | — | — | — | — | — | — | — | |
| Sleep-time ComputeBackbone=Gemma4-E4B, Use idle?=true2026.05 | 41.5 | 23.6 | 9 | — | — | — | — | — | — | — | — | — | |
| Sequential RevisionBackbone=Qwen3.5-4B, Use idle?=false2026.05 | 41.5 | 27.9 | 11.5 | — | — | — | — | — | — | — | — | — | |
| IdleSpecBackbone=Qwen3.5-4B, Use idle?=true2026.05 | 40.9 | 31.8 | 21.8 | — | — | — | — | — | — | — | — | — | |
| Sequential RevisionBackbone=Gemma4-E4B, Use idle?=false2026.05 | 39.6 | 26.4 | 9 | — | — | — | — | — | — | — | — | — | |
| VanillaBackbone=Qwen3.5-4B, Use idle?=false2026.05 | 37.7 | 26.4 | 11.5 | — | — | — | — | — | — | — | — | — | |
| VanillaBackbone=Gemma4-E4B, Use idle?=false2026.05 | 36.5 | 25.6 | 11.5 | — | — | — | — | — | — | — | — | — | |
| Sleep-time ComputeBackbone=Qwen3.5-4B, Use idle?=true2026.05 | 36.5 | 21.3 | 9 | — | — | — | — | — | — | — | — | — | |
| MiroFlowAgent Framework Category=Multi-Agents w. Tool + Memory, Base Model Series=Claude-series2025.12 | — | — | — | — | 74.5 | 82.4 | — | — | — | — | — | — |