General AI Assistant Tasks on GAIA Level 3 original (test)
37.5PerformanceClaude-4
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Claude-4Framework=CK-Pro, Setting=Manual2026.01 | 37.5 | 70.22 | 12.96 | |
| EvoRouteFramework=CK-Pro, Setting=Ours2026.01 | 33.33 | 25.3 | 14.35 | |
| Qwen3-14bFramework=Smolagent, Setting=Manual2026.01 | 27.39 | 0.1 | 4.4 | |
| EvoRouteFramework=Smolagent, Setting=Ours2026.01 | 27.04 | 5.7 | 1.2 | |
| GPT-4.1Framework=CK-Pro, Setting=Manual2026.01 | 26.77 | 9.62 | 4.71 | |
| PromptLLMFramework=CK-Pro, Setting=Routing2026.01 | 24.39 | 16.64 | 15.89 | |
| MasRouterFramework=CK-Pro, Setting=Routing2026.01 | 23.51 | 14.95 | 15 | |
| GraphRouterFramework=CK-Pro, Setting=Routing2026.01 | 22.94 | 15.5 | 14.22 | |
| Claude-4Framework=Smolagent, Setting=Manual2026.01 | 22.94 | 14.1 | 1.8 | |
| GPT-4.1Framework=Smolagent, Setting=Manual2026.01 | 16.01 | 3.5 | 2.1 | |
| GPT-4oFramework=CK-Pro, Setting=Manual2026.01 | 15.26 | 11.44 | 6.58 | |
| PromptLLMFramework=Smolagent, Setting=Routing2026.01 | 15.04 | 4.5 | 1.9 | |
| MasRouterFramework=Smolagent, Setting=Routing2026.01 | 15.04 | 4.5 | 1.8 | |
| GraphRouterFramework=Smolagent, Setting=Routing2026.01 | 15.03 | 4 | 1.8 | |
| Qwen3-14bFramework=CK-Pro, Setting=Manual2026.01 | 0 | 0.52 | 3.54 |