Vision-Language Navigation on EmbodiedBench Navigation latest (test)
86.7Success RateGPT4.1+VP
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT4.1+VPBackbone=GPT4.12025.12 | 86.7 | |
| VP+SRGPOBackbone=Qwen2.5-VL-3B-Ins2025.12 | 72.3 | |
| Claude-3.5-SonnetAccess=Close-Source2025.12 | 66.7 | |
| InternVL3-78BAccess=Open-Source2025.12 | 66.7 | |
| GPT4.1Access=Close-Source2025.12 | 65 | |
| Gemini-2.0-flashAccess=Close-Source2025.12 | 63.3 | |
| Ovis2-34BAccess=Open-Source2025.12 | 63.3 | |
| VP+GiGPO (w. VPR)Backbone=Qwen2.5-VL-3B-Ins2025.12 | 57.2 | |
| GPT-4oAccess=Close-Source2025.12 | 55 | |
| gemma-3-27b-itAccess=Open-Source2025.12 | 53.3 | |
| Claude-3.7-SonnetAccess=Close-Source2025.12 | 50 | |
| Qwen-VL-MaxAccess=Close-Source2025.12 | 50 | |
| Llama-3.2-11B-Vision-InsAccess=Open-Source2025.12 | 48.3 | |
| Qwen2.5-VL-72B-InsAccess=Open-Source2025.12 | 46.7 | |
| VP+GRPOBackbone=Qwen2.5-VL-3B-Ins2025.12 | 40.8 | |
| InternVL3-8BAccess=Open-Source2025.12 | 38.3 | |
| VP+SFTBackbone=Qwen2.5-VL-3B-Ins2025.12 | 36.7 | |
| GPT-4o-miniAccess=Close-Source2025.12 | 31.7 | |
| VP+GiGPO (vanilla)Backbone=Qwen2.5-VL-3B-Ins2025.12 | 29.4 | |
| Gemini-1.5-ProAccess=Close-Source2025.12 | 23.3 | |
| Llama-3.2-90B-Vision-InsAccess=Open-Source2025.12 | 23.3 | |
| Qwen2.5-VL-7B-InsAccess=Open-Source2025.12 | 20 | |
| Qwen2.5-VL-3B-InsAccess=Open-Source2025.12 | 16.7 |