Robot Manipulation on Google Robot variant evaluation SimplerEnv
98Pick Coke Success RateAsyncVLA
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| AsyncVLAProtocol=Visual Matching (M)2025.11 | 98 | 82.3 | 70.5 | 50.4 | 75.3 | |
| π0Protocol=Visual Matching (M)2025.11 | 97.9 | 78.7 | 62.3 | 46.6 | 71.4 | |
| ThinkActProtocol=Visual Matching (M)2025.11 | 92 | 72.4 | 50 | — | — | |
| AsyncVLAProtocol=Variant Aggregation (A)2025.11 | 91.6 | 81.7 | 58 | 26 | 64.3 | |
| π0Protocol=Variant Aggregation (A)2025.11 | 90.1 | 80.7 | 27.6 | 20.5 | 54.7 | |
| SpatialVLAProtocol=Variant Aggregation (A)2025.11 | 88 | 72.7 | 41.8 | 6.3 | 52.2 | |
| SpatialVLAProtocol=Visual Matching (M)2025.11 | 86 | 77.9 | 57.4 | 0 | 55.3 | |
| Discrete-Diffusion VLAProtocol=Visual Matching (M)2025.11 | 85.4 | 67.5 | 60.6 | — | — | |
| ThinkActProtocol=Variant Aggregation (A)2025.11 | 84 | 63.8 | 47.6 | — | — | |
| Discrete-Diffusion VLAProtocol=Variant Aggregation (A)2025.11 | 82.5 | 64.6 | 23.6 | — | — | |
| RT-2-XProtocol=Variant Aggregation (A)2025.11 | 82.3 | 79.2 | 35.3 | 20.6 | 54.4 | |
| RT-2-XProtocol=Visual Matching (M)2025.11 | 78.7 | 77.9 | 25 | 7.4 | 47.3 | |
| MolmoActProtocol=Visual Matching (M)2025.11 | 77.7 | 77.1 | 60 | — | — | |
| π0-FASTProtocol=Variant Aggregation (A)2025.11 | 77.6 | 68.2 | 31.3 | 0 | 44.3 | |
| RoboVLMProtocol=Visual Matching (M)2025.11 | 77.3 | 61.7 | 43.5 | 24.1 | 51.7 | |
| MolmoActProtocol=Variant Aggregation (A)2025.11 | 76.1 | 61.3 | 78.8 | — | — | |
| RoboVLMProtocol=Variant Aggregation (A)2025.11 | 75.6 | 60 | 10.6 | 0 | 36.6 | |
| π0-FASTProtocol=Visual Matching (M)2025.11 | 75.3 | 67.5 | 42.9 | 0 | 46.4 | |
| MagmaProtocol=Visual Matching (M)2025.11 | 75 | 53 | 58.9 | 8.3 | 48.8 | |
| MagmaProtocol=Variant Aggregation (A)2025.11 | 68.6 | 78.5 | 59 | 24 | 57.5 | |
| TraceVLAProtocol=Variant Aggregation (A)2025.11 | 60 | 56.4 | 31 | 0 | 36.9 | |
| TraceVLAProtocol=Visual Matching (M)2025.11 | 28 | 53.7 | 57 | 0 | 34.7 | |
| Octo-BaseProtocol=Visual Matching (M)2025.11 | 17 | 4.2 | 22.7 | 0 | 11 | |
| Octo-BaseProtocol=Variant Aggregation (A)2025.11 | 0.6 | 3.1 | 1.1 | 0 | 1.2 |