General AI assistant tasks on GAIA n=165 (dev)
73.93Average AccuracyOAgent
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| OAgentAgent Model=Claude-3-7, Paid Tools=Jina Reader, Whisper, Baidu & Bing API, Protocol=Pass@32025.08 | 73.93 | 83.02 | 74.42 | 53.85 | |
| ManusAgent Model=Claude, etc., Paid Tools=Unknown2025.08 | 73.3 | 86.5 | 70.1 | 57.7 | |
| Cognitive Kernel-ProAgent Model=Claude-3-7, Paid Tools=—, Protocol=Pass@32025.08 | 70.91 | 83.02 | 68.6 | 53.85 | |
| TraseAgentAgent Model=Claude, Paid Tools=Unknown2025.08 | 70.3 | 83.02 | 69.77 | 46.15 | |
| OWL-WorkforceAgent Model=Claude-3-7, Paid Tools=Whisper, o3-mini2025.08 | 69.09 | 84.91 | 67.44 | 42.31 | |
| Deep ResearchAgent Model=Unknown, Paid Tools=Unknown2025.08 | 67.36 | 74.29 | 69.06 | 47.6 | |
| OAgentAgent Model=Claude-3-7, Paid Tools=Jina Reader, Whisper, Baidu & Bing API2025.08 | 66.67 | 77.36 | 66.28 | 46.15 | |
| h2oGPTeAgent Model=Claude-3.5, Paid Tools=Unknown2025.08 | 63.64 | 67.92 | 67.44 | 42.31 | |
| Smolagents*Agent Model=Claude-3-7, Paid Tools=—, Protocol=Pass@32025.08 | 63.64 | 75.47 | 61.63 | 46.15 | |
| Cognitive Kernel-ProAgent Model=Claude-3-7, Paid Tools=—, Protocol=Voting2025.08 | 63.64 | 69.81 | 65.12 | 46.15 | |
| OWL-Workforce*Agent Model=Claude-3-7, Paid Tools=w/o whisper2025.08 | 60.61 | 73.58 | 62.79 | 26.92 | |
| OWL-RoleplayingAgent Model=4o & o3-mini, Paid Tools=Chunkr, FireCrawl, Whisper, o3-mini2025.08 | 58.18 | 81.14 | 54.65 | 23.08 | |
| Cognitive Kernel-ProAgent Model=Claude-3-7, Paid Tools=—2025.08 | 57.58 | 77.36 | 54.65 | 26.92 | |
| DesearchAgent Model=GPT-4o, Paid Tools=Unknown2025.08 | 56.97 | 71.7 | 58.14 | 23.08 | |
| TapeAgentsAgent Model=Claude-3-7, Paid Tools=—2025.08 | 55.76 | 71.7 | 53.49 | 30.77 | |
| AutoAgentAgent Model=Claude-3-5, Paid Tools=—2025.08 | 55.15 | 71.7 | 53.4 | 26.92 | |
| Smolagents*Agent Model=Claude-3-7, Paid Tools=—, Protocol=Voting2025.08 | 53.99 | 66.04 | 52.33 | 33.33 | |
| Smolagents*Agent Model=Claude-3-7, Paid Tools=—2025.08 | 52.1 | 60.38 | 54.65 | 26.92 | |
| SmolagentsAgent Model=Openai o1, Paid Tools=—2025.08 | 49.7 | 54.72 | 53.49 | 26.92 | |
| Magnetic-1Agent Model=OpenAI o1, Paid Tools=—2025.08 | 46.06 | 56.6 | 46.51 | 23.08 | |
| Cognitive Kernel-ProAgent Model=CK-Pro-8B, Paid Tools=—, Protocol=Pass@32025.08 | 38.18 | 50.94 | 38.37 | 11.54 | |
| Cognitive Kernel-ProAgent Model=CK-Pro-8B, Paid Tools=—, Protocol=Voting2025.08 | 34.54 | 47.17 | 33.72 | 11.54 | |
| Cognitive Kernel-ProAgent Model=CK-Pro-8B, Paid Tools=—2025.08 | 32.73 | 43.4 | 32.56 | 11.54 |