Embodied Planning on EB-Habitat (unseen)
57Avg Success RateGPT-4o
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GPT-4oType=Closed-Source General Model, Params=-2025.10 | 57 | 84 | 42 | 62 | 38 | 62 | 54 | |
| GPT-4.1Type=Closed-Source General Model, Params=-2025.10 | 50.67 | 90 | 38 | 50 | 36 | 46 | 44 | |
| Qwen2.5-VL-72B-InsType=Open-Source General Model, Params=72B2025.10 | 50.33 | 92 | 38 | 48 | 34 | 46 | 44 | |
| Qwen-VL-MaxType=Closed-Source General Model, Params=-2025.10 | 45.3 | 74 | 40 | 50 | 42 | 30 | 36 | |
| Gemini-2.0-flashType=Closed-Source General Model, Params=-2025.10 | 42.3 | 82 | 38 | 38 | 36 | 34 | 26 | |
| Llama-3.2-90B-Vision-InsType=Open-Source General Model, Params=90B2025.10 | 40.3 | 94 | 24 | 50 | 32 | 28 | 14 | |
| GPT-4o-miniType=Closed-Source General Model, Params=-2025.10 | 35 | 70 | 24 | 36 | 30 | 30 | 20 | |
| Gemma-3-12b-itsType=Open-Source General Model, Params=12B2025.10 | 23 | 58 | 10 | 24 | 18 | 24 | 4 | |
| RoboGPT-R1Type=Embodied Planning Model, Params=3B2025.10 | 22 | 64 | 8 | 18 | 20 | 12 | 10 | |
| REBPType=Embodied Planning Model, Params=7B2025.10 | 18.33 | 50 | 6 | 18 | 14 | 14 | 8 | |
| RoboBrainType=Embodied Planning Model, Params=7B2025.10 | 15.3 | 38 | 6 | 18 | 8 | 18 | 4 | |
| Qwen2.5-VL-7B-InsType=Open-Source General Model, Params=7B2025.10 | 15 | 42 | 6 | 22 | 12 | 4 | 4 | |
| Qwen2.5-VL-3B-InsType=Open-Source General Model, Params=3B2025.10 | 14.67 | 34 | 0 | 18 | 18 | 14 | 4 | |
| InternVL2.5-8BType=Open-Source General Model, Params=8B2025.10 | 11.3 | 36 | 4 | 0 | 10 | 16 | 2 | |
| TapaType=Embodied Planning Model, Params=7B2025.10 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |