Escape Room Solving on EscapeBench 1.0 (test)
4.33Hints UsedAverage Human
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Average HumanType=Human Performance2024.12 | 4.33 | 257.83 | 59.65 | — | — | |
| Claude-3.5-SonnetAgent Architecture=BaseAgent2024.12 | 8.97 | 690.31 | 28.95 | — | — | |
| GPT-4oAgent Architecture=BaseAgent2024.12 | 10.3 | 723.61 | 24.75 | — | — | |
| Gemini-1.5-proAgent Architecture=BaseAgent2024.12 | 11.06 | 824.31 | 24.18 | — | — | |
| Llama-3.1-70BAgent Architecture=BaseAgent2024.12 | 14.53 | 982.42 | 19 | — | — | |
| GPT-4o-miniAgent Architecture=BaseAgent2024.12 | 15.19 | 1,002.39 | 16.06 | — | — | |
| Qwen2.5-72BAgent Architecture=BaseAgent2024.12 | 16.5 | 1,102.5 | 12.46 | — | — | |
| Yi-1.5-34BAgent Architecture=BaseAgent2024.12 | 24 | 1,573.33 | 11.96 | — | — | |
| Ministral-8BAgent Architecture=BaseAgent2024.12 | 25.31 | 1,556.97 | 8.97 | — | — | |
| DeepSeek-LLM-67BAgent Architecture=BaseAgent2024.12 | 25.5 | 1,558.47 | 6.63 | — | — | |
| Llama-3.1-8BAgent Architecture=BaseAgent2024.12 | 25.86 | 1,543.3 | 10.1 | — | — | |
| Phi-3-medium-128kAgent Architecture=BaseAgent2024.12 | 32.19 | 1,871.19 | 7.34 | — | — | |
| Qwen2.5-7BAgent Architecture=BaseAgent2024.12 | 32.2 | 1,950.42 | 6.52 | — | — |