Long-horizon game playing on BALROG NetHack
6.8Progression RateGemini-3-Pro
Evaluation Results
| Method | Links | |
|---|---|---|
| Gemini-3-Procategory=Frontier proprietary (BALROG leaderboard)2026.07 | 6.8 | |
| Gemini-3.1-Pro-Thinkingcategory=Frontier proprietary (BALROG leaderboard)2026.07 | 2.6 | |
| Claude-Opus-4.5category=Frontier proprietary (BALROG leaderboard)2026.07 | 2 | |
| AUTOMEMbackbone=Qwen2.5-32B-Instruct, setup=memory training (loop #2)2026.07 | 1.85 | |
| Gemini-2.5-Procategory=Frontier proprietary (BALROG leaderboard)2026.07 | 1.7 | |
| AUTOMEMbackbone=Qwen2.5-32B-Instruct, setup=scaffold opt. (loop #1)2026.07 | 1.57 | |
| DeepSeek-R1category=Open-weight (BALROG leaderboard)2026.07 | 1.4 | |
| AUTOMEMbackbone=Qwen2.5-32B-Instruct, setup=memory-as-file-system, v02026.07 | 0.42 | |
| Qwen2.5-72B-Instructcategory=Open-weight (BALROG leaderboard)2026.07 | 0.3 | |
| Qwen2.5-7B-Instructcategory=Open-weight (BALROG leaderboard)2026.07 | 0 | |
| Qwen2.5-32B-Instructcontext-management method=sliding window2026.07 | 0 | |
| Qwen2.5-32B-Instructcontext-management method=sliding window + chain-of-thought2026.07 | 0 |