Failure Detection on Franka Panda DROID robot (unseen scenes)
94F1 (Move Banana)ROBOMETER
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| ROBOMETERZero-shot=true2026.03 | 94 | 91 | 83 | 58 | 76 | 73 | 90 | 81 | |
| RoboReward-4BZero-shot=true2026.03 | 91 | 80 | 73 | 40 | 57 | 73 | 95 | 74 | |
| T.U.Zero-shot=true2026.03 | 53 | 50 | 32 | 58 | 43 | 22 | 47 | 48 | |
| GPT-5-miniZero-shot=true2026.03 | 48 | 89 | 25 | 27 | 0 | 25 | 17 | 33 | |
| VLACZero-shot=true2026.03 | 45 | 0 | 0 | 16 | 0 | 0 | 0 | 16 |