Critic Accuracy on RoboCasa Overall v1
96.6Cab Countr AccuracyRobot Critics
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Robot CriticsTraining Data=Full (Success and Failure), Mode=Fine-tuned2026.06 | 96.6 | 90.8 | 96.5 | 91.6 | 93.6 | |
| Robot CriticsTraining Data=Success-Only, Mode=Fine-tuned2026.06 | 92.7 | 87.4 | 89.2 | 88.6 | 88.6 | |
| ROVERMode=Prompted (off-the-shelf)2026.06 | 70.7 | 79.8 | 72.7 | 71 | 72.1 | |
| Gemini 1.5 ProMode=Prompted (off-the-shelf)2026.06 | 70.6 | 42.8 | 65.9 | 39.8 | 54.7 | |
| GVLMode=Prompted (off-the-shelf)2026.06 | 52.8 | 58.4 | 57.6 | 53.8 | 53.1 | |
| Qwen2.5-72BMode=Prompted (off-the-shelf)2026.06 | 49.5 | 50.2 | 50.2 | 51.2 | 49.9 | |
| ProgressLMMode=Fine-tuned2026.06 | 20.6 | 70.9 | 85 | 77.4 | 61 |