SIMPLER
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
SIMPLER Visual Matching WidowX robot
95.8Put Spoon on Towel Score
51
SIMPLER Google Robot VA
98Pick Up Coke Can Success Rate
35
SIMPLER Visual Matching
78.1Average Success Rate
31
SIMPLER Google Robot Visual Matching
97.2PickCan Success Rate
29
SIMPLER
94.6Pick Coke Can Success Rate
23
SIMPLER WidowX + Bridge Setup
62.5Spoon Success Rate
22
SIMPLER Google Robot Setup
95.8Success Rate
12
SIMPLER Google Robot Setup
95.7Success Rate
12
SIMPLER
54.2Success Rate (StackG2Y)
11
SIMPLER Google robot, Visual Matching setting (test)
96Success Rate (PCC)
10
SIMPLER WidowX Environment
66.7Spoon on cloth (Part.)
9
SIMPLER Bridge
70.8Success Rate
9
SIMPLER Bridge tasks
62.5Success Rate
9
Simpler Google Robot tasks
98.3Pick Coke Can Success Rate
7
SIMPLER Overall
63Success Rate (All)
7
SIMPLER
81.5Success Rate (Google-VM)
6
SIMPLER Out-of-Distribution Env
51Success Count: Redbull on Plate
6
SIMPLER In-Distribution Env
52Carrot on Plate Success Rate
6
SIMPLER Google robot Cross-embodiment evaluation
56.3Spoon Put on Towel Success Rate
5
SIMPLER novel YCB and GSO objects 1.0 (test)
96Success Count: Green Cube
5
SIMPLER Benchmark novel deployment tasks
70Monster Energy Drink V1 Success Rate
4
SIMPLER (test)
26.3Success Rate
3