Calvin
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
CALVIN LH-MTLC
97.5Success Rate (1 Instruction)
10
DSMF-CALVIN (test)
90.64Accuracy
10
CALVIN ABC→D
98.9Success Rate (1 Task)
8
CALVIN ABC→D (Zero-shot)
96Completion Rate (Seq 1)
8
CALVIN ABC -> D
4.48Avg. Completed Tasks
8
CALVIN (unseen environment D)
96.41 Task Completion Rate
7
CALVIN D->D
96.4Success Rate (1 Task)
7
CALVIN ABCD -> D
4.67Average Completed Tasks
7
CALVIN
100Success Rate
7
CALVIN
100Success Rate
7
CALVIN
87.5Success Rate
7
CALVIN
96.6Success Rate
7
CALVIN
100Success Rate
7
Calvin ABC -> D
0.45Average Path Length
7
Calvin D -> D
2.92Average Length
7
CALVIN ABC → D (test)
43.7Success Rate (δ < 4)
7
CALVIN ABC-to-D transfer (test)
94.85Overall Success Rate (ST-SR)
6
CALVIN D->D
2.92Average Successful Length
6
CALVIN-5
95.2Score C1
5
CALVIN Hierarchical
95.53/5 Subtasks Success Rate
5
CALVIN Sequential
97Success Rate (3/5 Subtasks)
5
CALVIN (Unseen Views)
96.5Performance Score (View 1)
5
CALVIN Original View
93Success Rate
5
Calvin (val)
19.95PSNR
5
CALVIN
66.7Success Rate
4