Long-horizon game playing on BALROG MiniHack
40Progression RateGemini-3-Pro
Evaluation Results
| Method | Links | |
|---|---|---|
| Gemini-3-Procategory=Frontier proprietary (BALROG leaderboard)2026.07 | 40 | |
| AUTOMEMbackbone=Qwen2.5-32B-Instruct, setup=memory training (loop #2)2026.07 | 30 | |
| Gemini-3.1-Pro-Thinkingcategory=Frontier proprietary (BALROG leaderboard)2026.07 | 27.5 | |
| Claude-Opus-4.5category=Frontier proprietary (BALROG leaderboard)2026.07 | 27.5 | |
| AUTOMEMbackbone=Qwen2.5-32B-Instruct, setup=scaffold opt. (loop #1)2026.07 | 27.5 | |
| DeepSeek-R1category=Open-weight (BALROG leaderboard)2026.07 | 25 | |
| Gemini-2.5-Procategory=Frontier proprietary (BALROG leaderboard)2026.07 | 17.5 | |
| Qwen2.5-32B-Instructcontext-management method=sliding window + chain-of-thought2026.07 | 10 | |
| AUTOMEMbackbone=Qwen2.5-32B-Instruct, setup=memory-as-file-system, v02026.07 | 7.5 | |
| Qwen2.5-72B-Instructcategory=Open-weight (BALROG leaderboard)2026.07 | 5 | |
| Qwen2.5-32B-Instructcontext-management method=sliding window2026.07 | 2.5 | |
| Qwen2.5-7B-Instructcategory=Open-weight (BALROG leaderboard)2026.07 | 0 |