Robot Failure Analysis (MCQ) on RoboFAC Simulation
93FD ScoreKITE+Qwen2.5-7B+QLoRA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| KITE+Qwen2.5-7B+QLoRAFinetuned=true2026.04 | 93 | 69 | 92 | |
| RoboFAC-7BFinetuned=true2026.04 | 91 | 63 | 94 | |
| KITE + Qwen2.5-VL-7BFinetuned=false2026.04 | 88 | 44 | 55 | |
| GPT-4oFinetuned=false2026.04 | 64 | 21 | 71 | |
| Qwen2.5-VL-7BFinetuned=false2026.04 | 52 | 26 | 22 | |
| Gemini-2.0Finetuned=false2026.04 | 48 | 27 | 75 | |
| Qwen2.5-VL-3BFinetuned=false2026.04 | 38 | 4 | 51 |