Turn off lightbulb on CALVIN
100Success RateWAM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| WAMFine-tuning Protocol=Online Fine-Tuning, Model-based PPO Fine-Tuning Iterations=800, Total Physical Interactions=02026.03 | 100 | — | |
| BC+RLMethod=BC+RL, Future Condition=GTFuture, Training Seeds=32026.05 | 100 | — | |
| BC+RLMethod=BC+RL, Future Condition=GenFuture, Training Seeds=32026.05 | 94 | — | |
| BC+RLMethod=BC+RL, Future Condition=NoFuture, Training Seeds=32026.05 | 91.1 | — | |
| DiWAFine-tuning Protocol=Offline Fine-Tuning, Model-based PPO Fine-Tuning Iterations=800, Total Physical Interactions=02026.03 | 82.33 | — | |
| DiWAFine-tuning Protocol=Offline Fine-Tuning, Model-based PPO Fine-Tuning Iterations=800, Total Physical Interactions=02026.03 | 77.01 | — | |
| WAMFine-tuning Protocol=Online Fine-Tuning, Model-based PPO Fine-Tuning Iterations=800, Total Physical Interactions=02026.03 | 75.9 | — | |
| Diffusion PolicyFine-tuning Protocol=Base, Total Physical Interactions=-2026.03 | 68.8 | — | |
| Diffusion PolicyFine-tuning Protocol=Base, Total Physical Interactions=-2026.03 | 17.2 | — | |
| BC+RLMethod=BC+RL, Future Condition=WrongFuture, Training Seeds=32026.05 | 0 | — | |
| DPPOVision Encoder Type=Vision WM Encoder, Total Physical Interactions=~2.5M2026.03 | — | 327,066 | |
| DPPOVision Encoder Type=Vision, Total Physical Interactions=~8M2026.03 | — | 1,260,000 | |
| DPPOVision Encoder Type=Vision WM Encoder, Total Physical Interactions=~2.5M2026.03 | — | 277,333 | |
| DPPOVision Encoder Type=Vision, Total Physical Interactions=~8M2026.03 | — | 184,800 |