Failure reasoning and recovery on ManiSkill roll-out seeds (test)
90.94Binary SuccessFailSafe-VLM
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| FailSafe-VLM2025.10 | 90.94 | 83.68 | 0.6522 | |
| GPT-4o2025.10 | 70.07 | 19.6 | 0.0117 | |
| Gemini-2.5-flash2025.10 | 62.29 | 14.12 | -0.0121 | |
| Qwen2.5-VL2025.10 | 24.01 | 24.01 | 0 |