World Modelling on AlfWorld
61.5Exact Matchgemini-3-flash-preview-thinking
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| gemini-3-flash-preview-thinkingTask Type=clean2026.06 | 61.5 | 88.2 | 79.9 | 90.5 | |
| Looped World Model (Ours)Task Type=look, Number of parameters=about 1B2026.06 | 60.5 | 78.9 | 73.1 | 82 | |
| claude-opus-4-6-maxTask Type=look2026.06 | 60.5 | 73.2 | 68.9 | 78.6 | |
| Looped World Model (Ours)Task Type=clean, Number of parameters=about 1B2026.06 | 60.4 | 81.7 | 75 | 81.3 | |
| claude-opus-4-6-maxTask Type=clean2026.06 | 57.3 | 73.8 | 68.9 | 77.4 | |
| gemini-3-flash-preview-thinkingTask Type=look2026.06 | 55.8 | 86.6 | 74.4 | 97.7 | |
| Looped World Model (Ours)Task Type=heat, Number of parameters=about 1B2026.06 | 55 | 81.8 | 76.6 | 81.2 | |
| gemini-3-flash-preview-thinkingTask Type=heat2026.06 | 55 | 81.9 | 73.3 | 90.6 | |
| gemini-3-flash-preview-thinkingTask Type=cool2026.06 | 54.5 | 88.1 | 77 | 88.3 | |
| claude-opus-4-6-maxTask Type=Overall2026.06 | 53 | 72.6 | 66.8 | 77 | |
| claude-opus-4-6-maxTask Type=heat2026.06 | 52.5 | 67.6 | 64.4 | 74.1 | |
| Looped World Model (Ours)Task Type=Overall, Number of parameters=about 1B2026.06 | 51.6 | 80.4 | 71.6 | 81.1 | |
| claude-opus-4-6-maxTask Type=pick2026.06 | 51 | 72.8 | 65.7 | 78.1 | |
| Looped World Model (Ours)Task Type=cool, Number of parameters=about 1B2026.06 | 50 | 81.7 | 72.6 | 78.5 | |
| claude-opus-4-6-maxTask Type=cool2026.06 | 50 | 73 | 68.2 | 72.8 | |
| gemini-3-flash-preview-thinkingTask Type=Overall2026.06 | 50 | 83.5 | 71 | 90.2 | |
| Looped World Model (Ours)Task Type=pick, Number of parameters=about 1B2026.06 | 46.7 | 79.7 | 69.1 | 81.5 | |
| gemini-3-flash-preview-thinkingTask Type=pick2026.06 | 42.7 | 80.2 | 65.1 | 89.3 | |
| qwen-3.5-flashTask Type=clean2026.06 | 36.5 | 71.9 | 55.6 | 90.1 | |
| qwen-3.5-flashTask Type=look2026.06 | 27.9 | 66.2 | 46.3 | 92.8 | |
| qwen-3.5-flashTask Type=heat2026.06 | 27.5 | 70.1 | 49.5 | 91.2 | |
| qwen-3.5-flashTask Type=cool2026.06 | 27.3 | 72 | 52.6 | 85.1 | |
| qwen-3.5-flashTask Type=Overall2026.06 | 26 | 67.3 | 47.7 | 88.4 | |
| qwen-3.5-flashTask Type=pick2026.06 | 21.2 | 64.1 | 43.5 | 87.5 |