General AI Assistant Task on GAIA (accuracy, delta)
84.5AccuracyStep-3.5-Flash
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Step-3.5-FlashSize=196B-A11B, Context management=false2026.06 | 84.5 | — | |
| SearchSwarmSize=30B-A3B, Context management=true2026.06 | 82.5 | — | |
| MiroThinker-1.7-miniSize=30B-A3B, Context management=true2026.06 | 80.3 | — | |
| RedSearcherSize=30B-A3B, Context management=false2026.06 | 80.1 | — | |
| Seed-2.0-ProSize=-, Context management=false2026.06 | 78.6 | — | |
| LongSeekerSize=30B-A3B, Context management=true2026.06 | 77.7 | — | |
| GPT-5Size=-, Context management=false2026.06 | 76.4 | — | |
| MiniMax-M2Size=230B-A10B, Context management=false2026.06 | 75.7 | — | |
| DeepSeek V3.2Size=671B-A37B, Context management=false2026.06 | 75.1 | — | |
| Gemini-3.0-ProSize=-, Context management=false2026.06 | 74.8 | — | |
| MiroThinker-1.5-miniSize=30B-A3B, Context management=true2026.06 | 72 | — | |
| Claude-4.5-OpusSize=-, Context management=false2026.06 | 71.5 | — | |
| Tongyi DeepResearchSize=30B-A3B, Context management=false2026.06 | 70.9 | — | |
| Tongyi DR SwarmSize=30B-A3B, Context management=false2026.06 | 70.9 | — | |
| Claude-4.5-SonnetSize=-, Context management=false2026.06 | 66 | — | |
| Claude Opus 4.6Deployment=Cloud baseline2026.05 | 62 | — | |
| Qwen3.5-35BDeployment=Local model2026.05 | 55.7 | — | |
| Best LocalConfig=Oracle local routing frontier2026.05 | 55.7 | — | |
| Qwen3.5-122BDeployment=Local model2026.05 | 55 | — | |
| Gemma4-26BDeployment=Local model2026.05 | 52.1 | — | |
| Gemini 3.1 ProDeployment=Cloud baseline2026.05 | 51.3 | — | |
| Qwen3.5-27BDeployment=Local model2026.05 | 50.4 | — | |
| LLM-guided spec searchStudent Model=Qwen3.5-9B, Optimization Status=Optimized, Teacher Selection=Best teacher per benchmark2026.05 | 47.3 | 12.3 | |
| Nemotron-Super-120BDeployment=Local model2026.05 | 45 | — | |
| LLM-guided spec searchStudent Model=Gemma4-E4B, Optimization Status=Optimized, Teacher Selection=Best teacher per benchmark2026.05 | 38.6 | 10.4 | |
| Qwen3.5-9BStudent Model=Qwen3.5-9B, Optimization Status=Base2026.05 | 35 | — | |
| Qwen3.5-9BDeployment=Local model2026.05 | 35 | — | |
| GPT 5.4Deployment=Cloud baseline2026.05 | 33.3 | — | |
| LLM-guided spec searchStudent Model=Qwen3.5-4B, Optimization Status=Optimized, Teacher Selection=Best teacher per benchmark2026.05 | 28.7 | 6.7 | |
| Gemma4-E4BStudent Model=Gemma4-E4B, Optimization Status=Base2026.05 | 28.2 | — | |
| Gemma4-E4BDeployment=Local model2026.05 | 28.2 | — | |
| Qwen3.5-4BStudent Model=Qwen3.5-4B, Optimization Status=Base2026.05 | 22 | — | |
| LLM-guided spec searchStudent Model=Nemotron-Nano-4B, Optimization Status=Optimized, Teacher Selection=Best teacher per benchmark2026.05 | 16.1 | 8.1 | |
| Nemotron-Nano-4BStudent Model=Nemotron-Nano-4B, Optimization Status=Base2026.05 | 8 | — | |
| Granite 4.0 H-SmallDeployment=Local model2026.05 | 6.4 | — | |
| Granite 3.3 8BDeployment=Local model2026.05 | 4.2 | — |