Video Question Answering on ImplicitQA VRR-QA (val)
83Avg. AccuracyHuman (non-expert)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Human (non-expert)2026.05 | 83 | 85.6 | |
| ASC-MQRAM (margin threshold)=0, MQRA=true2026.06 | 73.23 | 78.69 | |
| ASC-MQRAM (margin threshold)=1, MQRA=true2026.06 | 73.23 | 78.48 | |
| ASC-MQRAM (margin threshold)=2, MQRA=true2026.06 | 72.83 | 78.28 | |
| ASCK (number of runs)=10, ASC=true2026.06 | 72.73 | 78.34 | |
| Single pass2026.06 | 68.41 | 74.6 | |
| GPT-o32026.05 | 64.1 | 68.6 | |
| Qwen3-VL-32B-AWQ (CoT) + self-consistency (k=5)chain-of-thought=true, quantization=4-bit AWQ, self-consistency=k=52026.05 | 51.3 | 54.5 | |
| Qwen3-VL-32B-AWQ (CoT)chain-of-thought=true, quantization=4-bit AWQ2026.05 | 48.7 | 52.5 | |
| Qwen3-VL-8B (CoT) + route (single↔dec.)chain-of-thought=true, decomposition=true, routing=single↔dec.2026.05 | 47.5 | 51.3 | |
| Qwen3-VL-8B (CoT)chain-of-thought=true2026.05 | 46.7 | 49.9 | |
| Qwen3-VL-8B (CoT) + decompositionchain-of-thought=true, decomposition=true2026.05 | 46.7 | 50.3 | |
| Qwen2.5-VL-7B (CoT)chain-of-thought=true2026.05 | 45.8 | 48.7 |