General AI Assistant Task on GAIA (test)
46.06Overall AccuracyMASLab-ReAct
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| MASLab-ReActBase Model=GPT-4.12025.05 | 46.06 | 56.6 | 18,278 | 47.67 | 35,636 | 19.23 | 43,525 | 31,303 | |
| OWL-RoleplayingBase Model=GPT-4.12025.05 | 33.94 | 43.4 | 48,073 | 30.23 | 101,827 | 26.92 | 101,986 | 84,586 | |
| OWL-RoleplayingBase Model=GPT-4.1-mini2025.05 | 26.67 | 35.85 | 51,543 | 25.58 | 58,881 | 11.54 | 107,635 | 64,206 | |
| MASLab-ReActBase Model=GPT-4.1-mini2025.05 | 26.67 | 33.96 | 19,866 | 26.74 | 41,768 | 11.54 | 55,743 | 36,935 | |
| DebateBase Model=GPT-4.12025.05 | 24.85 | 32.08 | 4,103 | 24.42 | 4,339 | 11.54 | 11,564 | 5,402 | |
| AgentVerseBase Model=GPT-4.1-mini2025.05 | 19.39 | 32.08 | 7,174 | 15.12 | 7,368 | 7.69 | 15,753 | 8,627 | |
| AgentVerseBase Model=GPT-4.12025.05 | 19.39 | 28.3 | 6,876 | 18.6 | 5,995 | 3.85 | 11,034 | 7,072 | |
| DebateBase Model=GPT-4.1-mini2025.05 | 17.58 | 24.53 | 4,388 | 16.28 | 4,870 | 7.69 | 12,972 | 5,992 | |
| SingleBase Model=GPT-4.12025.05 | 16.97 | 24.53 | 394 | 16.28 | 470 | 3.85 | 1,378 | 589 | |
| SCBase Model=GPT-4.12025.05 | 16.97 | 20.75 | 3,037 | 16.28 | 3,362 | 11.54 | 11,786 | 4,585 | |
| ReActModel Size=30B, Evaluation Samples=30, Evaluation Setting=single-pass2026.06 | 16.7 | — | — | — | — | — | — | — | |
| SCBase Model=GPT-4.1-mini2025.05 | 15.15 | 22.64 | 4,504 | 15.12 | 2,412 | 0 | 8,484 | 4,041 | |
| SingleBase Model=GPT-4.1-mini2025.05 | 13.94 | 16.98 | 663 | 16.28 | 353 | 0 | 1,529 | 638 | |
| RSEAModel Size=30B, Evaluation Samples=30, Evaluation Setting=single-pass2026.06 | 13.3 | — | — | — | — | — | — | — |