Retrieval on WalkieKnowledge
68.91Retrieval Accuracy@1Gemini 2.5 Pro
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemini 2.5 ProLatency (sec.)=~ 242025.10 | 68.91 | 88.08 | 89.12 | 40.94 | 56.97 | 58.13 | 68.91 | 34.54 | 21.56 | 68.9 | 77.8 | 78.1 | |
| Qwen 3.5+Latency (sec.)=~ 492025.10 | 66.32 | 83.42 | 87.56 | 41.15 | 64.34 | 69.78 | 66.32 | 40.59 | 27.67 | 66.3 | 74.4 | 75.4 | |
| VL-KnGVariant=GER-G, Latency (sec.)=~ 0.82025.10 | 65.8 | 81.35 | 83.94 | 38.54 | 67.53 | 71.75 | 65.81 | 45.6 | 29.84 | 65.8 | 72.9 | 73.5 | |
| VL-KnGVariant=GER-L, Latency (sec.)=~ 0.82025.10 | 61.66 | 80.31 | 85.49 | 35.28 | 65.39 | 71.11 | 61.66 | 44.21 | 29.22 | 62 | 70 | 72 | |
| VL-KnGVariant=GR, Latency (sec.)=~ 0.82025.10 | 53.16 | 62.11 | 64.21 | 28.28 | 49.11 | 52.32 | 52.63 | 34.91 | 22.52 | 53 | 57 | 57 | |
| Qwen 2.5 VLModel Size=72B2025.10 | 48.19 | 61.14 | 61.14 | 28.93 | 37.64 | 37.64 | 48.19 | 21.94 | 13.16 | 48 | 54 | 54 | |
| RoboHop2025.10 | 34.72 | 54.4 | 62.69 | 19.28 | 37.5 | 47.4 | 35.75 | 24.35 | 18.55 | 35 | 43 | 45 | |
| Qwen 2.5 VLModel Size=32B2025.10 | 32.12 | 68.91 | 70.47 | 16.38 | 40.56 | 42.09 | 32.13 | 25.22 | 15.85 | 32 | 49 | 49 | |
| WMNav2025.10 | 9.42 | 14.14 | 15.18 | 5.23 | 7.85 | 8.42 | 9.42 | 4.71 | 3.04 | 9 | 11 | 11 |