Route tracing on MapBench (val)
2.34Google ScoreGLM4.5V
Evaluation Results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GLM4.5Vtype=Pretrained Model2025.12 | 2.34 | 10.79 | 3.06 | 0.71 | 2.42 | 1.83 | 0.51 | 5.2 | 1.25 | 2.3 | 70.7 | |
| QwenVL-2.5 72Btype=Pretrained Model2025.12 | 2.12 | 0.73 | 0.9 | 1.01 | 1.19 | 1.34 | 1.05 | 0.81 | 1.71 | 1.32 | 42.9 | |
| Gemma3-27Btype=Pretrained Model2025.12 | 1.6 | 0.88 | 1.51 | 1.44 | 1.63 | 1.22 | 0.69 | 0.97 | 1.16 | 1.29 | 88.3 | |
| Gemma3-27B*type=Finetuned on MapTrace, training_dataset=MapTrace (synthetic)2025.12 | 1.36 | 1.03 | 1.02 | 1.18 | 1.3 | 1 | 0.83 | 1.53 | 1.13 | 1.13 | 94.7 | |
| Gemini 2.5 Protype=Pretrained Model2025.12 | 1.25 | 0.87 | 1.16 | 1.18 | 1.57 | 1.06 | 1.13 | 1.34 | 0.8 | 1.19 | 90.6 | |
| Gemini 2.5 Flashtype=Pretrained Model2025.12 | 1.2 | 0.77 | 0.84 | 1.81 | 1.71 | 1.28 | 0.88 | 1.84 | 1.3 | 1.29 | 98.1 | |
| Gemini 2.5 Flash*type=Finetuned on MapTrace, training_dataset=MapTrace (synthetic)2025.12 | 1.14 | 0.74 | 0.71 | 0.93 | 1.04 | 0.64 | 0.62 | 0.93 | 0.9 | 0.87 | 98.3 | |
| Mask-Methodtype=Baseline, learning=non-learning2025.12 | 0.91 | 0.75 | 0.67 | 0.91 | 0.68 | 0.65 | 0.64 | 0.78 | 0.9 | 0.79 | 73.4 |