Video Question Answering on RAVEN-QA Hard Queries
96.8AccuracyVLM Only
Evaluation Results
| Method | Links | |
|---|---|---|
| VLM OnlyVLM=Gp32026.06 | 96.8 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=Gp32026.06 | 95.1 | |
| RAVENEmbedder=QQMM-v2, VLM=Gp32026.06 | 92.7 | |
| RAVENEmbedder=SigLIP, VLM=Gp32026.06 | 85.4 | |
| VLM OnlyVLM=GPT-4o2026.06 | 85.4 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=GPT-52026.06 | 82.9 | |
| RAVENEmbedder=QQMM-v2, VLM=Gp2.52026.06 | 82.9 | |
| RAVENEmbedder=QQMM-v2, VLM=Gf2.52026.06 | 82.9 | |
| RAVENEmbedder=QQMM-v2, VLM=GPT-52026.06 | 82.9 | |
| RAVENEmbedder=CLIP-B, VLM=Gp32026.06 | 82.9 | |
| VLM OnlyVLM=Gp2.52026.06 | 82.9 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=Gf2.52026.06 | 80.5 | |
| RAVENEmbedder=SigLIP, VLM=Gp2.52026.06 | 80.5 | |
| RAVENEmbedder=SigLIP, VLM=GPT-52026.06 | 80.5 | |
| VLM OnlyVLM=GPT-52026.06 | 80.5 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=Gp2.52026.06 | 78 | |
| VLM OnlyVLM=Gf2.52026.06 | 78 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=GPT-4o2026.06 | 73.2 | |
| RAVENEmbedder=QQMM-v2, VLM=GPT-4o2026.06 | 73.2 | |
| RAVENEmbedder=SigLIP, VLM=GPT-4o2026.06 | 73.2 | |
| RAVENEmbedder=CLIP-B, VLM=GPT-52026.06 | 73.2 | |
| RAVENEmbedder=SigLIP, VLM=Gf2.52026.06 | 68.3 | |
| Caption (4o-mini)VLM=Gp32026.06 | 68.3 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=Embedder Only2026.06 | 63.4 | |
| RAVENEmbedder=QQMM-v2, VLM=Embedder Only2026.06 | 63.4 | |
| RAVENEmbedder=CLIP-B, VLM=Gp2.52026.06 | 63.4 | |
| Caption (4o-mini)VLM=Q3-32b2026.06 | 63.4 | |
| RAVENEmbedder=SigLIP, VLM=Embedder Only2026.06 | 61 | |
| Caption (4o-mini)VLM=GPT-52026.06 | 61 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=Q3-32b2026.06 | 58.5 | |
| Caption (4o-mini)VLM=Gp2.52026.06 | 58.5 | |
| Caption (4o-mini)VLM=Gf2.52026.06 | 58.5 | |
| RAVENEmbedder=SigLIP, VLM=Gm3-27b2026.06 | 56.1 | |
| RAVENEmbedder=CLIP-B, VLM=Gf2.52026.06 | 56.1 | |
| RAVENEmbedder=CLIP-B, VLM=GPT-4o2026.06 | 56.1 | |
| Caption (4o-mini)VLM=GPT-4o2026.06 | 56.1 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=Gm3-27b2026.06 | 53.7 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=Q2.5-32b2026.06 | 51.2 | |
| RAVENEmbedder=QQMM-v2, VLM=Gm3-27b2026.06 | 51.2 | |
| RAVENEmbedder=QQMM-v2, VLM=Q3-32b2026.06 | 48.8 | |
| RAVENEmbedder=SigLIP, VLM=Q2.5-32b2026.06 | 48.8 | |
| RAVENEmbedder=QQMM-v2, VLM=Q2.5-32b2026.06 | 46.3 | |
| RAVENEmbedder=CLIP-B, VLM=Gm3-27b2026.06 | 46.3 | |
| RAVENEmbedder=QQMM-v2, VLM=Gm3-12b2026.06 | 43.9 | |
| RAVENEmbedder=SigLIP, VLM=Gm3-12b2026.06 | 43.9 | |
| RAVENEmbedder=SigLIP, VLM=Q3-32b2026.06 | 41.5 | |
| Caption (4o-mini)VLM=Q2.5-32b2026.06 | 41.5 | |
| RAVENEmbedder=CLIP-B, VLM=Embedder Only2026.06 | 39 | |
| VLM OnlyVLM=Q2.5-32b2026.06 | 36.6 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=Gm3-12b2026.06 | 34.1 | |
| RAVENEmbedder=CLIP-B, VLM=Q3-32b2026.06 | 34.1 | |
| RAVENEmbedder=CLIP-B, VLM=Q2.5-32b2026.06 | 26.8 | |
| RAVENEmbedder=CLIP-B, VLM=Gm3-12b2026.06 | 24.4 | |
| VLM OnlyVLM=Gm3-27b2026.06 | 24.4 | |
| Caption (4o-mini)VLM=Gm3-27b2026.06 | 22 | |
| VLM OnlyVLM=Gm3-12b2026.06 | 19.5 | |
| Caption (4o-mini)VLM=Gm3-12b2026.06 | 17.1 | |
| Caption (4o-mini)VLM=Q2.5-7b2026.06 | 17.1 | |
| VLM OnlyVLM=Q3-32b2026.06 | 14.6 | |
| VLM OnlyVLM=Q2.5-7b2026.06 | 7.3 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=Q2.5-7b2026.06 | 4.9 | |
| RAVENEmbedder=QQMM-v2, VLM=Q2.5-7b2026.06 | 2.4 | |
| RAVENEmbedder=SigLIP, VLM=Q2.5-7b2026.06 | 2.4 | |
| RAVENEmbedder=CLIP-B, VLM=Q2.5-7b2026.06 | 2.4 |