Robot Manipulation on CALVIN ABC->D 1.0
96.8Success Rate (1 Inst)GR-MG
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| GR-MGFully-Annotated Data=100%, Partially-Annotated Data=false2024.08 | 96.8 | 89.3 | 81.5 | 72.7 | 64.4 | — | |
| GR-MG w/o progressFully-Annotated Data=100%, Partially-Annotated Data=false, Task Progress=Not provided2024.08 | 94.1 | 85 | 75.2 | 65.4 | 56.3 | — | |
| 3D Diff ActorFully-Annotated Data=100%, Partially-Annotated Data=false2024.08 | 93.8 | 80.3 | 66.2 | 53.3 | 41.2 | — | |
| GR-MG w/o textFully-Annotated Data=100%, Partially-Annotated Data=false, Condition=Image only2024.08 | 91.8 | 79.8 | 68.9 | 58.1 | 48.1 | — | |
| RoboDualInstruction Type=Free-form (GPT-4 enriched)2024.10 | 91.8 | 81.8 | 68.5 | 57.8 | 48.8 | 3.47 | |
| GR-MG w/o imageFully-Annotated Data=100%, Partially-Annotated Data=false, Condition=Text only2024.08 | 91 | 79.1 | 67.8 | 56.9 | 47.7 | — | |
| GR-MGFully-Annotated Data=10%, Partially-Annotated Data=true2024.08 | 90.3 | 74.5 | 61.2 | 47.4 | 37.5 | — | |
| SusieFully-Annotated Data=100%, Partially-Annotated Data=false2024.08 | 87 | 69 | 49 | 38 | 26 | 2.69 | |
| GR-1Fully-Annotated Data=100%, Partially-Annotated Data=false2024.08 | 85.4 | 71.2 | 59.6 | 49.7 | 40.1 | 3.06 | |
| RoboflamingoFully-Annotated Data=100%, Partially-Annotated Data=false2024.08 | 82.4 | 61.9 | 46.6 | 33.1 | 23.5 | 2.47 | |
| GR-MG w/o part. ann. dataFully-Annotated Data=10%, Partially-Annotated Data=false2024.08 | 82.4 | 60.8 | 42.2 | 28.7 | 19.7 | — | |
| SpilFully-Annotated Data=100%, Partially-Annotated Data=false2024.08 | 74.2 | 46.3 | 27.6 | 14.7 | 8 | 1.71 | |
| LCBInstruction Type=Free-form (GPT-4 enriched)2024.10 | 73.6 | 50.2 | 28.5 | 16 | 9.9 | 1.78 | |
| GR-1Fully-Annotated Data=10%, Partially-Annotated Data=false2024.08 | 67.2 | 37.1 | 19.8 | 10.8 | 6.9 | — | |
| 3D Diffuser ActorInstruction Type=Free-form (GPT-4 enriched)2024.10 | 65.2 | 39.1 | 20.3 | 11.7 | 6.1 | 1.42 | |
| RoboFlamingoInstruction Type=Free-form (GPT-4 enriched)2024.10 | 63 | 33 | 16.4 | 8.6 | 3.6 | 1.25 | |
| MDTFully-Annotated Data=100%, Partially-Annotated Data=false2024.08 | 61.7 | 40.6 | 23.8 | 14.7 | 8.7 | — | |
| HulcFully-Annotated Data=100%, Partially-Annotated Data=false2024.08 | 41.8 | 16.5 | 5.7 | 1.9 | 1.1 | — |