Navigation Question Answering on RAVEN-QA Habitat Simulation Tasks
88.75Overall AccuracyVLM Only
Evaluation Results
| Method | Links | |
|---|---|---|
| VLM OnlyVLM Backbone=Gemini-3-Pro2026.06 | 88.75 | |
| RAVEN (Seed)VLM Backbone=Gemini-3-Pro2026.06 | 86.84 | |
| RAVEN (QQMM-v2)VLM Backbone=Gemini-3-Pro2026.06 | 86.62 | |
| RAVEN (SigLIP)VLM Backbone=Gemini-3-Pro2026.06 | 86.2 | |
| RAVEN (Seed)VLM Backbone=Gemini-2.5-Flash2026.06 | 81.74 | |
| RAVEN (SigLIP)VLM Backbone=Gemini-2.5-Flash2026.06 | 81.1 | |
| RAVEN (QQMM-v2)VLM Backbone=Gemini-2.5-Flash2026.06 | 80.47 | |
| VLM OnlyVLM Backbone=GPT-5.22026.06 | 80.47 | |
| VLM OnlyVLM Backbone=Gemini-2.5-Flash2026.06 | 76.65 | |
| RAVEN (Seed)VLM Backbone=GPT-5.22026.06 | 76.22 | |
| RAVEN (QQMM-v2)VLM Backbone=GPT-5.22026.06 | 75.58 | |
| RAVEN (SigLIP)VLM Backbone=GPT-5.22026.06 | 71.97 | |
| RAVEN (Seed)Embedder Only=true2026.06 | 62.42 | |
| RAVEN (QQMM-v2)Embedder Only=true2026.06 | 61.78 | |
| RAVEN (SigLIP)Embedder Only=true2026.06 | 61.78 | |
| ReMEmbR (QQMM-v2)VLM Backbone=Gemini-3-Pro2026.06 | 61.36 | |
| RAVEN (Seed)VLM Backbone=Gemma3-27b2026.06 | 60.3 | |
| ReMEmbR (QQMM-v2)VLM Backbone=GPT-5.22026.06 | 57.32 | |
| RAVEN (SigLIP)VLM Backbone=Gemma3-27b2026.06 | 56.05 | |
| RAVEN (QQMM-v2)VLM Backbone=Gemma3-27b2026.06 | 54.99 | |
| ReMEmbR (QQMM-v2)VLM Backbone=Gemini-2.5-Flash2026.06 | 54.99 | |
| RAVEN (Seed)VLM Backbone=Qwen3-VL-32b2026.06 | 27.39 | |
| RAVEN (QQMM-v2)VLM Backbone=Qwen3-VL-32b2026.06 | 26.11 | |
| RAVEN (SigLIP)VLM Backbone=Qwen3-VL-32b2026.06 | 22.51 | |
| ReMEmbR (QQMM-v2)VLM Backbone=Qwen3-VL-32b2026.06 | 20.38 | |
| ReMEmbR (QQMM-v2)VLM Backbone=Gemma3-27b2026.06 | 15.71 | |
| VLM OnlyVLM Backbone=Gemma3-27b2026.06 | 11.04 | |
| VLM OnlyVLM Backbone=Qwen3-VL-32b2026.06 | 9.77 |