Long-horizon task completion on CALVIN
93.8Success Rate (1 Task)3D Diffuser Actor
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| 3D Diffuser ActorTraining Data=Lang, Evaluation Protocol=Zero-shot2024.11 | 93.8 | 80.3 | 66.2 | 53.3 | 41.2 | 3.35 | |
| VidManTraining Data=Lang, Evaluation Protocol=Zero-shot2024.11 | 91.5 | 76.4 | 68.2 | 59.2 | 46.7 | 3.42 | |
| SuSIETrain episodes=All2024.02 | 87 | 69 | 49 | 38 | 26 | 2.69 | |
| SuSIETraining Data=All, Evaluation Protocol=Zero-shot2024.11 | 87 | 69 | 49 | 38 | 26 | 2.69 | |
| GR-1Train episodes=Lang2024.02 | 85.4 | 71.2 | 59.6 | 49.7 | 40.1 | 3.06 | |
| GR-1Training Data=Lang, Evaluation Protocol=Zero-shot2024.11 | 85.4 | 71.2 | 59.6 | 49.7 | 40.1 | 3.06 | |
| RoboFlamingoTrain episodes=Lang2024.02 | 82.4 | 61.9 | 46.6 | 33.1 | 23.5 | 2.48 | |
| RoboFlamingoTraining Data=Lang, Evaluation Protocol=Zero-shot2024.11 | 82.4 | 61.9 | 46.6 | 33.1 | 23.5 | 2.48 | |
| RT-1Train episodes=Lang2024.02 | 53.3 | 22.2 | 9.4 | 3.8 | 1.3 | 0.9 | |
| RT-1Training Data=Lang, Evaluation Protocol=Zero-shot2024.11 | 53.3 | 22.2 | 9.4 | 3.8 | 1.3 | 0.9 | |
| HULCTrain episodes=All2024.02 | 41.8 | 16.5 | 5.7 | 1.9 | 1.1 | 0.67 | |
| HULCTraining Data=All, Evaluation Protocol=Zero-shot2024.11 | 41.8 | 16.5 | 5.7 | 1.9 | 1.1 | 0.67 | |
| MCILTrain episodes=All2024.02 | 30.4 | 1.3 | 0.2 | 0 | 0 | 0.31 | |
| MCILTraining Data=All, Evaluation Protocol=Zero-shot2024.11 | 30.4 | 1.3 | 0.2 | 0 | 0 | 0.31 | |
| 3D Diffusion PolicyTraining Data=Lang, Evaluation Protocol=Zero-shot2024.11 | 28.7 | 2.7 | 0 | 0 | 0 | 0.31 |