Spatial Navigation and Reasoning on RedMaze 23K
58.72Route Planning Completion Rate (RP.CR)GPT-4
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GPT-4Model Category=API Models, Variant=-2026.04 | 58.72 | 41.37 | 57.1 | 29.29 | 44.27 | |
| DeepSeek-V3Model Category=API Models, Variant=-2026.04 | 51.38 | 25 | 49.96 | 38.35 | 21.75 | |
| Qwen-2.5-VL-32B + SFT + DPOModel Scale=32B Series, Variant=+ SFT + DPO2026.04 | 47.63 | 26.74 | 32.78 | 25.47 | 19.96 | |
| Qwen-2.5-VL-32B + SFT + SDPO (STAR)Model Scale=32B Series, Variant=+ SFT + SDPO (STAR)2026.04 | 46.42 | 29.27 | 35 | 26.87 | 20.14 | |
| Qwen-2.5-VL-32B + SFTModel Scale=32B Series, Variant=+ SFT2026.04 | 45.12 | 27.1 | 32.09 | 22.45 | 19.81 | |
| Qwen-2.5-VL-32BModel Scale=32B Series, Variant=Base2026.04 | 44.8 | 26.23 | 31.11 | 17.92 | 17.23 | |
| Qwen-2.5-VL-7B + SFT + SDPO (STAR)Model Scale=7B Series, Variant=+ SFT + SDPO (STAR)2026.04 | 33.61 | 13.01 | 26.32 | 21.42 | 18.41 | |
| Llama-3.1-8BModel Category=Open-Source Baselines, Variant=-2026.04 | 28.43 | 6.92 | 17.75 | 17.63 | 16.14 | |
| DeepSeek-R1-14BModel Category=Open-Source Baselines, Variant=-2026.04 | 24.88 | 4.15 | 15.2 | 13.42 | 15.58 | |
| Qwen-2.5-VL-7B + SFTModel Scale=7B Series, Variant=+ SFT2026.04 | 23.42 | 7.51 | 26.79 | 22.92 | 18.17 | |
| Qwen-2.5-VL-7BModel Scale=7B Series, Variant=Base2026.04 | 21.58 | 6.9 | 26.11 | 18.72 | 18.04 | |
| Qwen-2.5-VL-7B + SFT + DPOModel Scale=7B Series, Variant=+ SFT + DPO2026.04 | 15.96 | 4.48 | 31.12 | 21.84 | 18.32 |