VLABench
Benchmarks
TaskDatasetSOTA resultTrendResults
VLABench
64.83Success Rate
35
VLABench vlm_evaluation 1.0 (test)
85.8AUC
25
VLABench
46.3Average Success Rate
19
VLABench (Unseen Tasks)
71.3bACC
12
VLABench (Seen Tasks)
85.6Balanced Accuracy (bACC)
12
VLABench Avg.
65.9Progress Score (PS)
10
VLABench Instruction
70.2Progress Score (PS)
10
VLABench Commonsense
57.3Progress Score (PS)
10
VLABench Cross Category
61Primitive Success (PS)
10
VLABench In-dist.
81.1Progress Score (PS)
10
VLABench Texture
62.3Progress Score (PS)
9
VLABench
78In Distribution Accuracy
9
VLABench seen object configurations π0.5
38.8Success Rate
8
VLABench T2
39.9Success Rate
8
VLABench T1
64.8Success Rate (SR)
8
VLABench GR00T-N1.6 (seen object configurations)
35.7Success Rate
5
VLABench Average (test)
71.2IS
5
VLABench Unseen Texture (test)
76IS
5
VLABench Semantic Instruction (test)
82IS Score
5
VLABench Common Sense (test)
74.2IS
5
VLABench Cross-Category (test)
54.1IS
5
VLABench Real-world (Average)
55Success Rate (SR)
5
VLABench Real-world (Long Horizon Tier)
38Success Rate (SR)
5
VLABench Real-world (Semantic Tier)
42Success Rate (SR)
5
VLABench Real-world (Basic Tier)
97Success Rate (SR)
5