General AI Assistants Evaluation on GAIA n=50
0.5944AccuracyGPT-5 search
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-5 searchWeb search=true, Trials per question=64, Sample size (n)=50, Random seed=422025.12 | 0.5944 | — | 0.183 | 0.745 | |
| Claude 4.5 SonnetWeb search=true, Trials per question=64, Sample size (n)=50, Random seed=422025.12 | 0.3971 | — | 0.184 | 0.756 | |
| Claude 4.5 HaikuWeb search=true, Trials per question=64, Sample size (n)=50, Random seed=422025.12 | 0.3226 | — | 0.164 | 0.738 | |
| Gemini 2.5 ProWeb search=false, Trials per question=64, Sample size (n)=50, Random seed=422025.12 | 0.3128 | — | 0.18 | 0.772 | |
| Deepseek-v3p1Web search=false, Trials per question=64, Sample size (n)=50, Random seed=422025.12 | 0.2228 | — | 0.123 | 0.699 | |
| GPT-4o searchWeb search=true, Trials per question=64, Sample size (n)=50, Random seed=422025.12 | 0.2159 | — | 0.115 | 0.671 | |
| Qwen3-235b-a22bWeb search=false, Trials per question=64, Sample size (n)=50, Random seed=422025.12 | 0.1297 | — | 0.085 | 0.736 |