Robotic Manipulation on RoboTwin-MeM 1.0 (test)
62Rearrange Blocks Hard (n=1) Success RateEventVLA (VA only)
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| EventVLA (VA only)Memory Category=End-to-end Memory-based, Memory Module=VA only2026.06 | 62 | 13 | 5 | 20 | 0 | 26 | 0 | 18 | 18 | |
| EventVLA (VA+KEM)Memory Category=End-to-end Memory-based, Memory Module=VA+KEM2026.06 | 62 | 93 | 90 | 54 | 94 | 63 | 98 | 48 | 75.2 | |
| EventVLA (w/o NMS)Post-processing=None2026.06 | 62 | 93 | 49 | 36 | 10 | 35 | 97 | 45 | 53.4 | |
| EventVLA (hard label)Label Type=Hard2026.06 | 59 | 77 | 28 | 62 | 85 | 36 | 6 | 37 | 48.8 | |
| EventVLA (implicit memory bank)Memory Bank Type=Implicit2026.06 | 51 | 9 | 16 | 37 | 1 | 68 | 2 | 15 | 24.9 | |
| EventVLA (Nmax = 2)Buffer Capacity (Nmax)=22026.06 | 51 | 35 | 28 | 33 | 39 | 53 | 0 | 17 | 32 | |
| MemoryVLA (QwenOFT)Memory Category=End-to-end Memory-based, Backbone=QwenOFT2026.06 | 39 | 0 | 1 | 9 | 1 | 11 | 0 | 25 | 10.8 | |
| MemERMemory Category=Dual-system Memory-based2026.06 | 32 | 4 | 12 | 2 | 0 | 26 | 3 | 5 | 10.5 | |
| EventVLA (chunk size=30)Execution Chunk Size=302026.06 | 22 | 98 | 18 | 28 | 16 | 29 | 0 | 38 | 31.1 | |
| π0.5Memory Category=Non Memory-based2026.06 | 20 | 19 | 1 | 14 | 0 | 8 | 0 | 0 | 7.8 | |
| EventVLA (chunk size=15)Execution Chunk Size=152026.06 | 16 | 30 | 2 | 17 | 6 | 16 | 10 | 12 | 13.6 | |
| MemoryVLA (OpenVLA)Memory Category=End-to-end Memory-based, Backbone=OpenVLA2026.06 | 12 | 0 | 0 | 1 | 0 | 10 | 2 | 14 | 4.9 | |
| QwenOFTMemory Category=Non Memory-based2026.06 | 3 | 26 | 0 | 0 | 0 | 0 | 0 | 1 | 3.8 | |
| Mem-0Memory Category=Dual-system Memory-based2026.06 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |