Navigation Visual Question Answering on NaVQA Short (S) horizon
76.2Descriptive AccuracyRAVEN
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| RAVENLLM=Gemini-3-Pro, Encoder=Seed1.6-Embed2026.06 | 76.2 | 4.58 | 0.2 | |
| RAVENLLM=Gemini-3-Pro, Encoder=QQMM-v2, Quantization=4-bit2026.06 | 76.2 | 2.37 | 0.23 | |
| RAVENLLM=Gemini-2.5-Pro, Encoder=QQMM-v2, Quantization=4-bit2026.06 | 69 | 3.79 | 0.29 | |
| RAVENLLM=Gemini-2.5-Pro, Encoder=SigLIP2026.06 | 66.7 | 7.06 | 0.24 | |
| VLM OnlyLLM=Gemini-2.5-Pro2026.06 | 66.7 | 3.9 | 0.11 | |
| ReMEmbRLLM=Gemini-2.5-Pro, Encoder=MixedBread2026.06 | 66.7 | 6.11 | 0.16 | |
| VLM OnlyLLM=Gemini-3-Pro2026.06 | 66.7 | 4.81 | 0.21 | |
| ReMEmbRLLM=Gemini-3-Pro, Encoder=MixedBread2026.06 | 66.7 | 2.78 | 0.04 | |
| RAVENLLM=GPT-4o, Encoder=Seed1.6-Embed2026.06 | 64.7 | 4.49 | 0.21 | |
| RAVENLLM=Gemini-2.5-Flash, Encoder=SigLIP2026.06 | 63.4 | 7.39 | 0.24 | |
| VLM OnlyLLM=GPT-4o2026.06 | 63.4 | 7.4 | 17.61 | |
| RAVENLLM=Gemini-2.5-Pro, Encoder=Seed1.6-Embed2026.06 | 62.9 | 15.37 | 0.34 | |
| RAVENLLM=GPT-4o, Encoder=QQMM-v2, Quantization=4-bit2026.06 | 62.5 | 3.99 | 0.19 | |
| RAVENLLM=Gemini-2.5-Pro, Encoder=CLIP-B2026.06 | 61.9 | 12.35 | 0.24 | |
| ReMEmbRLLM=Gemini-2.5-Flash, Encoder=MixedBread2026.06 | 59.5 | 9.1 | 0.2 | |
| VLM OnlyLLM=Gemini-2.5-Flash2026.06 | 58.5 | 7.78 | 0.19 | |
| RAVENLLM=Gemini-2.5-Flash, Encoder=QQMM-v2, Quantization=4-bit2026.06 | 51.2 | 4.49 | 0.21 | |
| RAVENLLM=Gemini-2.5-Flash, Encoder=CLIP-B2026.06 | 51.2 | 9.84 | 0.21 | |
| RAVENLLM=Gemini-2.5-Flash, Encoder=Seed1.6-Embed2026.06 | 41.7 | 25.31 | 0.33 | |
| ReMEmbRLLM=Gemini-2.5-Flash, Encoder=SigLIP2026.06 | 31 | 17.45 | 0.59 | |
| ReMEmbRLLM=Gemini-2.5-Flash, Encoder=CLIP-B2026.06 | 28.6 | 19.8 | 0.5 | |
| ReMEmbRLLM=Gemini-2.5-Flash, Encoder=Seed1.6-Embed2026.06 | 26.2 | 21.29 | 0.25 | |
| ReMEmbRLLM=Gemini-2.5-Flash, Encoder=QQMM-v2, Quantization=4-bit2026.06 | 19 | 9.46 | 0.43 |