Counting on RoboMME Counting
96Bin Fill Success RateHuman Performance
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Human Performance2026.03 | 96 | 100 | 80 | 78 | |
| MME-VLA (SimpleSG)Memory Representation Type=Symbolic Memory, Integration Mechanism=Oracle2026.03 | 85.78 | 99.78 | 100 | 44.67 | |
| MME-VLA (GroundSG)Memory Representation Type=Symbolic Memory, Integration Mechanism=Oracle2026.03 | 85.78 | 100 | 100 | 49.67 | |
| MME-VLA (SimpleSG)Memory Representation Type=Symbolic Memory, Integration Mechanism=QwenVL2026.03 | 77.56 | 95.33 | 5.11 | 0.44 | |
| MME-VLA (FrameSamp)Memory Representation Type=Perceptual Memory, Integration Mechanism=Expert2026.03 | 57.33 | 86.22 | 94.67 | 28.89 | |
| MemER2026.03 | 56.67 | 79.33 | 59.33 | 0 | |
| MME-VLA (TokenDrop)Memory Representation Type=Perceptual Memory, Integration Mechanism=Expert2026.03 | 54.22 | 87.56 | 91.78 | 4.22 | |
| MME-VLA (GroundSG)Memory Representation Type=Symbolic Memory, Integration Mechanism=QwenVL2026.03 | 52 | 92.67 | 7.33 | 0 | |
| MME-VLA (TokenDrop)Memory Representation Type=Perceptual Memory, Integration Mechanism=Context2026.03 | 48.67 | 85.11 | 94.67 | 3.11 | |
| MME-VLA (SimpleSG)Memory Representation Type=Symbolic Memory, Integration Mechanism=Gemini2026.03 | 46 | 63 | 45 | 2 | |
| MME-VLA (FrameSamp)Memory Representation Type=Perceptual Memory, Integration Mechanism=Context2026.03 | 41.22 | 72 | 73.67 | 13.67 | |
| SAM2Act+2026.03 | 40 | 76 | 25.33 | 0 | |
| MME-VLA (FrameSamp)Memory Representation Type=Perceptual Memory, Integration Mechanism=Modul2026.03 | 39.56 | 87.33 | 92 | 42 | |
| MME-VLA (RMT)Memory Representation Type=Recurrent Memory, Integration Mechanism=Expert2026.03 | 35.78 | 60.22 | 36 | 5.56 | |
| MME-VLA (TTT)Memory Representation Type=Recurrent Memory, Integration Mechanism=Context2026.03 | 35.56 | 62.89 | 42.44 | 3.33 | |
| MME-VLA (TTT)Memory Representation Type=Recurrent Memory, Integration Mechanism=Expert2026.03 | 34.89 | 63.78 | 41.33 | 4 | |
| MME-VLA (TokenDrop)Memory Representation Type=Perceptual Memory, Integration Mechanism=Modul2026.03 | 34.44 | 83.56 | 86 | 5.33 | |
| MME-VLA (TTT)Memory Representation Type=Recurrent Memory, Integration Mechanism=Modul2026.03 | 34.22 | 65.11 | 36.67 | 2.11 | |
| MME-VLA (RMT)Memory Representation Type=Recurrent Memory, Integration Mechanism=Modul2026.03 | 33.33 | 60.78 | 37.78 | 4.67 | |
| MME-VLA (RMT)Memory Representation Type=Recurrent Memory, Integration Mechanism=Context2026.03 | 32.44 | 56.89 | 33.56 | 5.78 | |
| pi_0.52026.03 | 30 | 42.89 | 35.56 | 6.67 | |
| pi_0.5past actions=true2026.03 | 26.67 | 58.33 | 26.67 | 4.67 | |
| MME-VLA (GroundSG)Memory Representation Type=Symbolic Memory, Integration Mechanism=Gemini2026.03 | 26 | 18 | 4 | 3 |