Video Question Answering on RAVEN-QA Simple Queries
100AccuracyRAVEN
Evaluation Results
| Method | Links | |
|---|---|---|
| RAVENEmbedder=Seed1.6-Embed, VLM=Gp32026.06 | 100 | |
| VLM OnlyVLM=Gp32026.06 | 100 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=Gf2.52026.06 | 98.1 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=GPT-52026.06 | 98.1 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=GPT-4o2026.06 | 98.1 | |
| RAVENEmbedder=QQMM-v2, VLM=Gp32026.06 | 98.1 | |
| RAVENEmbedder=QQMM-v2, VLM=Gp2.52026.06 | 98.1 | |
| RAVENEmbedder=SigLIP, VLM=Gp32026.06 | 98.1 | |
| RAVENEmbedder=SigLIP, VLM=Gp2.52026.06 | 98.1 | |
| RAVENEmbedder=CLIP-B, VLM=GPT-52026.06 | 98.1 | |
| VLM OnlyVLM=Gp2.52026.06 | 98.1 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=Gp2.52026.06 | 96.3 | |
| RAVENEmbedder=QQMM-v2, VLM=Gf2.52026.06 | 96.3 | |
| RAVENEmbedder=QQMM-v2, VLM=GPT-52026.06 | 96.3 | |
| RAVENEmbedder=QQMM-v2, VLM=GPT-4o2026.06 | 96.3 | |
| RAVENEmbedder=CLIP-B, VLM=Gp2.52026.06 | 96.3 | |
| VLM OnlyVLM=Gf2.52026.06 | 96.3 | |
| Caption (4o-mini)VLM=Gp2.52026.06 | 96.3 | |
| RAVENEmbedder=SigLIP, VLM=Gf2.52026.06 | 94.4 | |
| RAVENEmbedder=SigLIP, VLM=GPT-52026.06 | 94.4 | |
| RAVENEmbedder=SigLIP, VLM=GPT-4o2026.06 | 94.4 | |
| RAVENEmbedder=CLIP-B, VLM=Gp32026.06 | 94.4 | |
| RAVENEmbedder=CLIP-B, VLM=Gf2.52026.06 | 94.4 | |
| VLM OnlyVLM=GPT-4o2026.06 | 94.4 | |
| RAVENEmbedder=QQMM-v2, VLM=Q2.5-32b2026.06 | 92.6 | |
| RAVENEmbedder=QQMM-v2, VLM=Q3-32b2026.06 | 92.6 | |
| Caption (4o-mini)VLM=Gf2.52026.06 | 92.6 | |
| RAVENEmbedder=QQMM-v2, VLM=Embedder Only2026.06 | 90.7 | |
| Caption (4o-mini)VLM=GPT-52026.06 | 90.7 | |
| Caption (4o-mini)VLM=Q3-32b2026.06 | 90.7 | |
| RAVENEmbedder=CLIP-B, VLM=GPT-4o2026.06 | 88.9 | |
| Caption (4o-mini)VLM=GPT-4o2026.06 | 88.9 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=Gm3-12b2026.06 | 87 | |
| Caption (4o-mini)VLM=Gp32026.06 | 87 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=Embedder Only2026.06 | 85.2 | |
| RAVENEmbedder=QQMM-v2, VLM=Gm3-27b2026.06 | 85.2 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=Q2.5-32b2026.06 | 83.3 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=Q3-32b2026.06 | 83.3 | |
| RAVENEmbedder=SigLIP, VLM=Embedder Only2026.06 | 83.3 | |
| VLM OnlyVLM=GPT-52026.06 | 83.3 | |
| RAVENEmbedder=QQMM-v2, VLM=Gm3-12b2026.06 | 81.5 | |
| RAVENEmbedder=SigLIP, VLM=Q2.5-32b2026.06 | 79.6 | |
| RAVENEmbedder=CLIP-B, VLM=Gm3-27b2026.06 | 77.8 | |
| RAVENEmbedder=SigLIP, VLM=Gm3-27b2026.06 | 75.9 | |
| RAVENEmbedder=SigLIP, VLM=Gm3-12b2026.06 | 75.9 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=Gm3-27b2026.06 | 74.1 | |
| RAVENEmbedder=CLIP-B, VLM=Gm3-12b2026.06 | 74.1 | |
| RAVENEmbedder=CLIP-B, VLM=Embedder Only2026.06 | 74.1 | |
| RAVENEmbedder=SigLIP, VLM=Q3-32b2026.06 | 72.2 | |
| RAVENEmbedder=CLIP-B, VLM=Q3-32b2026.06 | 72.2 | |
| Caption (4o-mini)VLM=Q2.5-32b2026.06 | 68.5 | |
| RAVENEmbedder=CLIP-B, VLM=Q2.5-32b2026.06 | 59.3 | |
| Caption (4o-mini)VLM=Q2.5-7b2026.06 | 59.3 | |
| RAVENEmbedder=Seed1.6-Embed, VLM=Q2.5-7b2026.06 | 31.5 | |
| RAVENEmbedder=SigLIP, VLM=Q2.5-7b2026.06 | 31.5 | |
| VLM OnlyVLM=Q2.5-32b2026.06 | 29.6 | |
| Caption (4o-mini)VLM=Gm3-12b2026.06 | 27.8 | |
| VLM OnlyVLM=Q3-32b2026.06 | 25.9 | |
| RAVENEmbedder=QQMM-v2, VLM=Q2.5-7b2026.06 | 22.2 | |
| RAVENEmbedder=CLIP-B, VLM=Q2.5-7b2026.06 | 18.5 | |
| Caption (4o-mini)VLM=Gm3-27b2026.06 | 18.5 | |
| VLM OnlyVLM=Gm3-12b2026.06 | 14.8 | |
| VLM OnlyVLM=Gm3-27b2026.06 | 13 | |
| VLM OnlyVLM=Q2.5-7b2026.06 | 13 |