Tool-use Agent on HLE-500
26.1Text-Only PerformanceAutoAgent
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| AutoAgentBackbone=gemini-3-pro2026.03 | 26.1 | 17.7 | 24.2 | |
| AutoAgentBackbone=gemini-3-pro-thinking2026.03 | 23.5 | 17.7 | 22.2 | |
| Direct AnswerBackbone=gemini-3-pro-thinking2026.03 | 20.4 | 15.6 | 19.4 | |
| Direct AnswerBackbone=gemini-3-pro2026.03 | 20.4 | 14.2 | 19 | |
| DeepAgentBackbone=gemini-3-pro-thinking2026.03 | 15.8 | 8.8 | 14.2 | |
| ReActBackbone=gemini-3-pro-thinking2026.03 | 12.4 | 11.5 | 12.2 | |
| DeepAgentBackbone=gemini-3-pro2026.03 | 12.1 | 4.4 | 10.4 | |
| ReActBackbone=gemini-3-pro2026.03 | 8.5 | 9.7 | 8.8 | |
| AutoAgentBackbone=gpt-4o2026.03 | 5.4 | 7.1 | 5.8 | |
| DeepAgentBackbone=gpt-4o2026.03 | 4.9 | 5.3 | 5 | |
| ReActBackbone=gpt-4o2026.03 | 3.4 | 5.3 | 3.8 | |
| Direct AnswerBackbone=gpt-4o2026.03 | 2.8 | 4.4 | 3.2 |