Scientific Agent Task on ScienceAgentBench (test)
18.6Success Rate (SR)SciNav
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SciNavBase Model=GPT-4o (2024-11-20), debug steps=32026.03 | 18.6 | 69.9 | 0.342 | |
| SciNavBase Model=GPT-4o (2024-05-13), debug steps=32026.03 | 16.1 | 66 | 0.512 | |
| Self-DebugBase Model=GPT-4o (2024-11-20), debug steps=102026.03 | 15 | 67 | 0.03 | |
| Self-DebugBase Model=GPT-4o (2024-05-13), debug steps=102026.03 | 14.7 | 71.2 | 0.057 | |
| OpenHandsBase Model=GPT-4o (2024-05-13)2026.03 | 13.1 | 62.8 | 1.093 | |
| Direct PromptingBase Model=GPT-4o (2024-05-13)2026.03 | 7.5 | 42.2 | 0.011 |