List your company
ResearchFormerly WizWand

Navigation Question Answering on RAVEN-QA Habitat Simulation Tasks

88.75Overall AccuracyVLM Only
6.610827.935449.2670.5846Jun 23, 2026
Updated 4mo ago

Evaluation results

MethodLinks
VLM OnlyVLM Backbone=Gemini-3-Pro2026.0688.75
RAVEN (Seed)VLM Backbone=Gemini-3-Pro2026.0686.84
RAVEN (QQMM-v2)VLM Backbone=Gemini-3-Pro2026.0686.62
RAVEN (SigLIP)VLM Backbone=Gemini-3-Pro2026.0686.2
RAVEN (Seed)VLM Backbone=Gemini-2.5-Flash2026.0681.74
RAVEN (SigLIP)VLM Backbone=Gemini-2.5-Flash2026.0681.1
RAVEN (QQMM-v2)VLM Backbone=Gemini-2.5-Flash2026.0680.47
VLM OnlyVLM Backbone=GPT-5.22026.0680.47
VLM OnlyVLM Backbone=Gemini-2.5-Flash2026.0676.65
RAVEN (Seed)VLM Backbone=GPT-5.22026.0676.22
RAVEN (QQMM-v2)VLM Backbone=GPT-5.22026.0675.58
RAVEN (SigLIP)VLM Backbone=GPT-5.22026.0671.97
RAVEN (Seed)Embedder Only=true2026.0662.42
RAVEN (QQMM-v2)Embedder Only=true2026.0661.78
RAVEN (SigLIP)Embedder Only=true2026.0661.78
ReMEmbR (QQMM-v2)VLM Backbone=Gemini-3-Pro2026.0661.36
RAVEN (Seed)VLM Backbone=Gemma3-27b2026.0660.3
ReMEmbR (QQMM-v2)VLM Backbone=GPT-5.22026.0657.32
RAVEN (SigLIP)VLM Backbone=Gemma3-27b2026.0656.05
RAVEN (QQMM-v2)VLM Backbone=Gemma3-27b2026.0654.99
ReMEmbR (QQMM-v2)VLM Backbone=Gemini-2.5-Flash2026.0654.99
RAVEN (Seed)VLM Backbone=Qwen3-VL-32b2026.0627.39
RAVEN (QQMM-v2)VLM Backbone=Qwen3-VL-32b2026.0626.11
RAVEN (SigLIP)VLM Backbone=Qwen3-VL-32b2026.0622.51
ReMEmbR (QQMM-v2)VLM Backbone=Qwen3-VL-32b2026.0620.38
ReMEmbR (QQMM-v2)VLM Backbone=Gemma3-27b2026.0615.71
VLM OnlyVLM Backbone=Gemma3-27b2026.0611.04
VLM OnlyVLM Backbone=Qwen3-VL-32b2026.069.77