Robotic Manipulation on RLBench 10 tasks
99Take Umbrella Success RatePointACT
Evaluation results
| Method | Links | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PointACTModel size=3B, Episodes=1002026.05 | 99 | — | — | — | — | — | — | — | — | — | 82.3 | 91 | 99 | 96 | 59 | 81 | 99 | 69 | 90 | 40 | |
| PolarNetSetup=Multi-task single-variation2023.09 | 97.8 | 97.8 | 86 | 99.6 | 80.5 | 94.1 | 100 | 93.4 | 80.5 | 68.1 | 89.8 | — | — | — | — | — | — | — | — | — | |
| PolarNetSetup=Single-task single-variation2023.09 | 97.5 | 96.7 | 91.9 | 99.9 | 82.5 | 96.1 | 99.9 | 93.5 | 94.1 | 68.6 | 92.1 | — | — | — | — | — | — | — | — | — | |
| ACT3DEpisodes=1002026.05 | 97 | — | — | — | — | — | — | — | — | — | 64.5 | 94 | 62 | 99 | 80 | 87 | 53 | 31 | 31 | 11 | |
| HiveformerLearning Protocol=Single-task learning2022.09 | 90.1 | 92.2 | 77.1 | 99.6 | 69.7 | 96.2 | 100 | 95.4 | 81.9 | 82.1 | 88.4 | — | — | — | — | — | — | — | — | — | |
| HiveformerSetup=Single-task single-variation2023.09 | 90.1 | 92.2 | 77.1 | 99.6 | 69.7 | 96.2 | 100 | 95.4 | 81.9 | 82.1 | 88.4 | — | — | — | — | — | — | — | — | — | |
| HiveformerLearning Protocol=Multi-task learning2022.09 | 89.2 | 88.9 | 92.9 | 100 | 75.3 | 58.2 | 100 | 78.7 | 71.2 | 79.1 | 83.3 | — | — | — | — | — | — | — | — | — | |
| HiveformerSetup=Multi-task single-variation2023.09 | 89.2 | 88.9 | 92.9 | 100 | 75.3 | 58.2 | 100 | 78.7 | 71.2 | 79.1 | 83.3 | — | — | — | — | — | — | — | — | — | |
| Auto-λLearning Protocol=Multi-task learning2022.09 | 80 | 87 | 78 | 95 | 31 | 62 | 100 | 77 | 19 | 64 | 69.3 | — | — | — | — | — | — | — | — | — | |
| Auto-λSetup=Multi-task single-variation2023.09 | 80 | 87 | 78 | 95 | 31 | 62 | 100 | 77 | 19 | 64 | 69.3 | — | — | — | — | — | — | — | — | — | |
| EO1Model size=3B, Episodes=1002026.05 | 76 | — | — | — | — | — | — | — | — | — | 73.2 | 97 | 99 | 100 | 95 | 83 | 46 | 40 | 61 | 35 | |
| HiveformerLearning Protocol=Multi-task learning, Instruction condition=w/o instructions2022.09 | 73 | 83.8 | 13.9 | 97 | 41.9 | 54.3 | 98.9 | 36.2 | 68.5 | 74.1 | 64.2 | — | — | — | — | — | — | — | — | — | |
| ARMLearning Protocol=Single-task learning2022.09 | 70 | 70 | 80 | — | — | — | 100 | — | 70 | — | — | — | — | — | — | — | — | — | — | — | |
| CogACTModel size=7B, Episodes=202026.05 | 60 | — | — | — | — | — | — | — | — | — | 60 | 80 | 85 | 90 | 65 | 90 | 50 | 35 | 25 | 25 | |
| HybridVLAModel size=7B, Episodes=202026.05 | 50 | — | — | — | — | — | — | — | — | — | 74 | 85 | 95 | 100 | 90 | 100 | 50 | 70 | 50 | 50 | |
| Auto-λLearning Protocol=Single-task learning2022.09 | 37 | 82 | 72 | 95 | 36 | 31 | 100 | 36 | 23 | 38 | 55 | — | — | — | — | — | — | — | — | — | |
| Auto-λSetup=Single-task single-variation2023.09 | 37 | 82 | 72 | 95 | 36 | 31 | 100 | 36 | 23 | 38 | 55 | — | — | — | — | — | — | — | — | — | |
| OpenVLAModel size=7B, Episodes=202026.05 | 35 | — | — | — | — | — | — | — | — | — | 41 | 65 | 40 | 75 | 60 | 80 | 20 | 15 | 10 | 10 | |
| π0Model size=2.6B, Episodes=202026.05 | 35 | — | — | — | — | — | — | — | — | — | 55 | 90 | 60 | 100 | 30 | 90 | 25 | 75 | 5 | 45 | |
| ManipLLMModel size=7B, Episodes=202026.05 | 10 | — | — | — | — | — | — | — | — | — | 38 | 50 | 80 | 40 | 20 | 80 | 35 | 25 | 15 | 20 | |
| ARMSetup=Single-task single-variation2023.09 | — | 70 | 80 | — | — | — | 100 | — | 70 | 70 | — | — | — | — | — | — | — | — | — | — |