Video Question Answering on VideoMME (Aspect Metrics)
30.9VYRBaseline+CoT+SFT
Evaluation results
| Method | Links | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Baseline+CoT+SFTBackbone=Qwen3-VL-8B-Instruct, Chain-of-Thought=true, Supervised Fine-Tuning=true2026.04 | 30.9 | — | — | — | — | 40.7 | 42.2 | 3.892 | 55.9 | 15.8 | 60.4 | 35.4 | 0.466 | 1.162 | 1.313 | 1.753 | 1.307 | 53.9 | |
| Baseline+SFTBackbone=Qwen3-VL-8B-Instruct, Supervised Fine-Tuning=true2026.04 | 20.7 | — | — | — | — | 60.2 | 62.3 | 3.573 | 29.7 | 20.2 | 64.8 | 37.6 | 1.322 | 0.492 | 1.454 | 1.747 | 1.441 | 51.6 | |
| Baseline (Qwen3-VL-8B-Instruct)Backbone=Qwen3-VL-8B-Instruct2026.04 | 19.3 | — | — | — | — | 61.7 | 63.7 | 3.571 | 28.4 | 21.1 | 65.6 | 39 | 1.376 | 0.483 | 1.709 | 1.447 | 0.837 | 31 | |
| Baseline+CoTBackbone=Qwen3-VL-8B-Instruct, Chain-of-Thought=true2026.04 | 19 | — | — | — | — | 56 | 57.4 | 3.543 | 29 | 14.6 | 61.1 | 38.4 | 1.272 | 1.066 | 1.971 | 1.601 | 0.212 | 44.1 | |
| Internvl3.5-VL-241BParameters=241B2026.04 | 14.5 | — | — | — | — | 57.3 | 60.5 | 3.356 | 38.4 | 39.7 | 64.5 | 25.3 | 1.189 | 0.469 | 1.919 | 1.386 | 1.469 | 51.5 | |
| Qwen3-VL-30BParameters=30B2026.04 | 13.8 | — | — | — | — | 58.7 | 62.1 | 3.707 | 34.7 | 38.8 | 63.8 | 27.1 | 1.211 | 0.628 | 1.924 | 1.54 | 1.123 | 50.4 | |
| Qwen3-VL-235BParameters=235B2026.04 | 12.7 | — | — | — | — | 55.6 | 57.3 | 3.274 | 42.6 | 40.7 | 60.4 | 26.1 | 1.147 | 0.341 | 1.832 | 1.476 | -1.316 | 53.7 | |
| VTHM-MoE2026.04 | 4.1 | — | — | — | — | 39.8 | 41.7 | 3.618 | 42.6 | 25.6 | 50.2 | 8.2 | 0.31 | 0.885 | 0.362 | 1.815 | 0.773 | 61.4 | |
| Gemini-3.1-Pro2026.04 | 0.7 | — | — | — | — | 13.5 | 13.2 | 3.387 | 60.1 | 11.4 | 28.7 | 1.6 | 0.045 | 0.388 | 0.698 | 1.336 | 0.212 | 59.8 | |
| LLoViModel Reasoner=GPT-4 (1.8T)2025.01 | — | 52.2 | 55.3 | 51.8 | 50.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| video-grounded entailment tree reasoningModel Reasoner=VideoLLAVA (7B)2025.01 | — | 55.9 | 53.8 | 54 | 50.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VideoTreeModel Reasoner=GPT-4 (1.8T)2025.01 | — | 55.7 | 54.3 | 54.2 | 52.6 | — | — | — | — | — | — | — | — | — | — | — | — | — |