Multi-step reasoning and tool use on GAIA (val)
80Pass@1AORCHESTRA
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| AORCHESTRAModel Setup=Gemini-3-Flash2026.02 | 80 | 86.06 | 71.62 | |
| AORCHESTRAModel Setup=DeepSeek-V3.22026.02 | 67.87 | 80 | 58.43 | |
| OpenHandsModel Setup=Gemini-3-Flash2026.02 | 66.06 | 72.73 | 48.49 | |
| OpenHandsModel Setup=DeepSeek-V3.22026.02 | 63.64 | 72.12 | 48.35 | |
| AORCHESTRAModel Setup=Claude-4.5-haiku2026.02 | 60.61 | 73.9 | 55.44 | |
| Mini-SWEModel Setup=Gemini-3-Flash2026.02 | 58.18 | 68.48 | 49.49 | |
| OpenHandsModel Setup=Claude-4.5-haiku2026.02 | 54.55 | 61.21 | 45.13 | |
| Mini-SWEModel Setup=DeepSeek-V3.22026.02 | 50.3 | 63.63 | 54.76 | |
| ReActModel Setup=Gemini-3-Flash2026.02 | 49.09 | 66.06 | 47.22 | |
| ReActModel Setup=Claude-4.5-haiku2026.02 | 47.88 | 62.42 | 43.62 | |
| ReActModel Setup=DeepSeek-V3.22026.02 | 46.7 | 71.51 | 38.23 | |
| Mini-SWEModel Setup=Claude-4.5-haiku2026.02 | 40.61 | 60 | 36.3 |