Video Evidence Retrieval on WalkieKnowledge original
68.91Retrieval Accuracy@1Gemini 1.5
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemini 1.5Variant=Pro2026.06 | 68.91 | 88.08 | 89.12 | 40.94 | 56.97 | 58.13 | 68.91 | 34.54 | 21.56 | 68.9 | 77.8 | 78.1 | |
| VL-MemKnG2026.06 | 67.69 | 73.85 | 77.95 | 40.05 | 59.41 | 63.5 | 67.69 | 39.14 | 26.46 | 67.7 | 70.6 | 71.5 | |
| Qwen 2.52026.06 | 66.32 | 83.42 | 87.56 | 41.15 | 64.34 | 69.78 | 66.32 | 40.59 | 27.67 | 66.3 | 74.4 | 75.4 | |
| VL-KnGVariant=GER-G2026.06 | 65.8 | 81.35 | 83.94 | 38.54 | 67.53 | 71.75 | 65.81 | 45.6 | 29.84 | 65.8 | 72.9 | 73.5 | |
| VL-KnGVariant=GER-L2026.06 | 61.66 | 80.31 | 85.49 | 35.28 | 65.39 | 71.11 | 61.66 | 44.21 | 29.22 | 61.7 | 70.4 | 71.6 | |
| Gemini 1.5Variant=Flash2026.06 | 61.66 | 82.9 | 83.94 | 37.76 | 53.61 | 54.72 | 61.66 | 32.12 | 19.69 | 61.7 | 71.2 | 71.4 | |
| VL-KnGVariant=GR2026.06 | 53.16 | 62.11 | 64.21 | 28.28 | 49.11 | 52.32 | 52.63 | 34.91 | 22.52 | 52.6 | 56.8 | 57.2 | |
| RoboHop2026.06 | 34.72 | 54.4 | 62.69 | 19.28 | 37.5 | 47.4 | 35.75 | 24.35 | 18.55 | 34.7 | 43.4 | 45.4 |