World Modelling (QA/Text Metrics) on ScienceWorld
100EM (Exact Match)Looped World Model
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Looped World ModelTask Type=Lifespan, Number of Parameters=1B, Number of consecutive actions=52026.06 | 100 | 100 | 100 | 100 | — | — | |
| Looped World ModelTask Type=Conductivity, Number of Parameters=1B, Number of consecutive actions=52026.06 | 87 | 89 | 87.8 | 87.9 | — | — | |
| Looped World ModelTask Type=Thermometer, Number of Parameters=1B, Number of consecutive actions=52026.06 | 83.3 | 85.3 | 83.3 | 83.3 | — | — | |
| claude-opus-4-6-maxTask Type=Thermometer, Number of consecutive actions=52026.06 | 83.3 | 98.1 | 93.5 | 97.2 | — | — | |
| Looped World ModelTask Type=StateChange, Number of Parameters=1B, Number of consecutive actions=52026.06 | 80 | 83.1 | 80 | 80 | — | — | |
| Looped World ModelTask Type=Genetics, Number of Parameters=1B, Number of consecutive actions=52026.06 | 78.3 | 80.2 | 78.9 | 79.8 | — | — | |
| Looped World ModelTask Type=Find, Number of Parameters=1B, Number of consecutive actions=52026.06 | 76.9 | 90.4 | 82.7 | 85.8 | — | — | |
| Looped World ModelTask Type=Grow, Number of Parameters=1B, Number of consecutive actions=52026.06 | 73.8 | 80 | 75.5 | 79.8 | — | — | |
| Looped World ModelTask Type=Melt, Number of Parameters=1B, Number of consecutive actions=52026.06 | 73 | 91.9 | 85.7 | 91.6 | — | — | |
| claude-opus-4-6-maxTask Type=Grow, Number of consecutive actions=52026.06 | 70.8 | 81.6 | 76.1 | 80.7 | — | — | |
| claude-opus-4-6-maxTask Type=Find, Number of consecutive actions=52026.06 | 69.2 | 83.8 | 78.9 | 84.6 | — | — | |
| Looped World ModelTask Type=Overall, Number of Parameters=1B, Number of consecutive actions=52026.06 | 68.4 | — | — | 83.9 | 85.3 | 80.7 | |
| Looped World ModelTask Type=Boil, Number of Parameters=1B, Number of consecutive actions=52026.06 | 66.7 | 79 | 75.3 | 77.5 | — | — | |
| Looped World ModelTask Type=Incline, Number of Parameters=1B, Number of consecutive actions=52026.06 | 59.3 | 95.3 | 90.4 | 93.4 | — | — | |
| claude-opus-4-6-maxTask Type=Genetics, Number of consecutive actions=52026.06 | 59.2 | 71.8 | 65.7 | 71.6 | — | — | |
| Looped World ModelTask Type=Power, Number of Parameters=1B, Number of consecutive actions=52026.06 | 57.1 | 63.9 | 60.8 | 61.5 | — | — | |
| claude-opus-4-6-maxTask Type=Conductivity, Number of consecutive actions=52026.06 | 47.8 | 67.2 | 53.1 | 72.7 | — | — | |
| claude-opus-4-6-maxTask Type=Overall, Number of consecutive actions=52026.06 | 47.2 | — | — | 72.3 | 72.8 | 64.4 | |
| Looped World ModelTask Type=Chemistry, Number of Parameters=1B, Number of consecutive actions=52026.06 | 44.4 | 64.4 | 54.2 | 57.9 | — | — | |
| claude-opus-4-6-maxTask Type=Chemistry, Number of consecutive actions=52026.06 | 44.4 | 59.8 | 46.3 | 59.2 | — | — | |
| claude-opus-4-6-maxTask Type=Power, Number of consecutive actions=52026.06 | 42.9 | 47.3 | 45.3 | 45.8 | — | — | |
| claude-opus-4-6-maxTask Type=StateChange, Number of consecutive actions=52026.06 | 40 | 65.5 | 44.6 | 73.3 | — | — | |
| claude-opus-4-6-maxTask Type=Melt, Number of consecutive actions=52026.06 | 36.5 | 52.4 | 46.3 | 53.4 | — | — | |
| claude-opus-4-6-maxTask Type=Incline, Number of consecutive actions=52026.06 | 34 | 86.5 | 76.2 | 83.8 | — | — | |
| Looped World ModelTask Type=Freeze, Number of Parameters=1B, Number of consecutive actions=52026.06 | 25 | 59.7 | 31.2 | 54.8 | — | — | |
| claude-opus-4-6-maxTask Type=Boil, Number of consecutive actions=52026.06 | 22.2 | 33.3 | 30.2 | 32.3 | — | — | |
| claude-opus-4-6-maxTask Type=Freeze, Number of consecutive actions=52026.06 | 12.5 | 33.6 | 21.6 | 37.3 | — | — | |
| Looped World ModelTask Type=LifeStages, Number of Parameters=1B, Number of consecutive actions=52026.06 | 0 | 18.3 | 6.1 | 10.2 | — | — | |
| claude-opus-4-6-maxTask Type=LifeStages, Number of consecutive actions=52026.06 | 0 | 10.6 | 6 | 6 | — | — | |
| claude-opus-4-6-maxTask Type=Lifespan, Number of consecutive actions=52026.06 | 0 | 61.4 | 0 | 58.3 | — | — |