Navigation Question Answering on RAVEN-QA Real-World Robot Tasks
97.14Overall AccuracyRAVEN (SigLIP)
Evaluation Results
| Method | Links | |
|---|---|---|
| RAVEN (SigLIP)VLM Backbone=GPT-5.22026.06 | 97.14 | |
| RAVEN (QQMM-v2)VLM Backbone=GPT-5.22026.06 | 93.33 | |
| RAVEN (Seed)VLM Backbone=GPT-5.22026.06 | 93.33 | |
| VLM OnlyVLM Backbone=GPT-5.22026.06 | 92.86 | |
| RAVEN (QQMM-v2)VLM Backbone=Gemini-2.5-Flash2026.06 | 92.38 | |
| RAVEN (Seed)VLM Backbone=Gemini-3-Pro2026.06 | 90.47 | |
| RAVEN (QQMM-v2)VLM Backbone=Gemini-3-Pro2026.06 | 90 | |
| RAVEN (SigLIP)VLM Backbone=Gemini-3-Pro2026.06 | 90 | |
| VLM OnlyVLM Backbone=Gemini-3-Pro2026.06 | 86.19 | |
| RAVEN (Seed)VLM Backbone=Gemini-2.5-Flash2026.06 | 85.71 | |
| VLM OnlyVLM Backbone=Gemini-2.5-Flash2026.06 | 85.24 | |
| RAVEN (SigLIP)VLM Backbone=Gemini-2.5-Flash2026.06 | 84.76 | |
| ReMEmbR (QQMM-v2)VLM Backbone=Gemini-3-Pro2026.06 | 80.48 | |
| ReMEmbR (QQMM-v2)VLM Backbone=Gemini-2.5-Flash2026.06 | 76.67 | |
| ReMEmbR (QQMM-v2)VLM Backbone=GPT-5.22026.06 | 74.76 | |
| RAVEN (QQMM-v2)VLM Backbone=Gemma3-27b2026.06 | 67.62 | |
| RAVEN (QQMM-v2)Embedder Only=true2026.06 | 66.67 | |
| RAVEN (Seed)Embedder Only=true2026.06 | 66.67 | |
| RAVEN (SigLIP)VLM Backbone=Gemma3-27b2026.06 | 47.62 | |
| RAVEN (Seed)VLM Backbone=Gemma3-27b2026.06 | 45.71 | |
| RAVEN (SigLIP)Embedder Only=true2026.06 | 42.86 | |
| RAVEN (SigLIP)VLM Backbone=Qwen3-VL-32b2026.06 | 35.24 | |
| RAVEN (Seed)VLM Backbone=Qwen3-VL-32b2026.06 | 31.9 | |
| RAVEN (QQMM-v2)VLM Backbone=Qwen3-VL-32b2026.06 | 30 | |
| ReMEmbR (QQMM-v2)VLM Backbone=Qwen3-VL-32b2026.06 | 28.57 | |
| ReMEmbR (QQMM-v2)VLM Backbone=Gemma3-27b2026.06 | 14.76 | |
| VLM OnlyVLM Backbone=Gemma3-27b2026.06 | 14.29 | |
| VLM OnlyVLM Backbone=Qwen3-VL-32b2026.06 | 4.76 |