Video Question Answering on ImplicitQA VRR-QA (test)
92.95Average AccuracyTest final pipeline
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Test final pipelineSetting=Test final pipeline, Questions=1722026.06 | 92.95 | 93.79 | |
| Gemini 3.1 Pro + SC (k=5)self-consistency=k=52026.05 | 81.2 | 78.9 | |
| ASCK (number of runs)=10, ASC=true2026.06 | 81.16 | 80.91 | |
| Prior best2026.05 | 80.9 | — | |
| ASC-MQRAM (margin threshold)=0, MQRA=true2026.06 | 80.85 | 80.33 | |
| ASC-MQRAM (margin threshold)=1, MQRA=true2026.06 | 80.85 | 80.33 | |
| ASC-MQRAM (margin threshold)=2, MQRA=true2026.06 | 78.09 | 77.36 | |
| Single pass2026.06 | 72.71 | 73.62 | |
| Qwen3-VL-32B-AWQ (CoT) + self-consistency (k=5)chain-of-thought=true, quantization=4-bit AWQ, self-consistency=k=52026.05 | 58.5 | 56.2 | |
| Qwen3-VL-8B (CoT) + route (single↔dec.)chain-of-thought=true, decomposition=true, routing=single↔dec.2026.05 | 56.5 | 54.8 | |
| Qwen3-VL-8B (CoT) + decompositionchain-of-thought=true, decomposition=true2026.05 | 56.2 | 53.1 | |
| Qwen3-VL-32B-AWQ (CoT)chain-of-thought=true, quantization=4-bit AWQ2026.05 | 55.9 | 56.2 | |
| Qwen3-VL-8B (CoT)chain-of-thought=true2026.05 | 54.6 | 52.3 | |
| Qwen2.5-VL-7B (CoT)chain-of-thought=true2026.05 | 50.2 | 47.3 |