Video Question Answering on Video-MME
86.9AccuracyGemini-3.0-Flash
Evaluation Results
| Method | Links | |
|---|---|---|
| Gemini-3.0-FlashZero-shot evaluation=true2026.01 | 86.9 | |
| Gemini-2.5-ProZero-shot evaluation=true2026.01 | 85.2 | |
| Qwen3-VL-8B + GRPOParams=8B, Training Scale=5.3K QA, Zero-shot evaluation=true2026.01 | 76.78 | |
| Full PrecisionModel=Qwen2.5-VL-72B-Instruct, Weight bits=16 bit, Memory=136.74 GB, Evaluation protocol=zero-shot2026.07 | 73.3 | |
| Qwen3-VL-8B (baseline)Params=8B, Training Scale=1.0T tokens2026.01 | 71.4 | |
| Full PrecisionModel=Qwen2.5-VL-32B-Instruct, Weight bits=16 bit, Memory=62.31 GB, Evaluation protocol=zero-shot2026.07 | 70.5 | |
| Qwen2-VL-7B + GRPOParams=7B, Training Scale=5.3K QA, Zero-shot evaluation=true2026.01 | 69.12 | |
| Qwen2-VL-7B (baseline)Params=7B, Training Scale=1.2T tokens2026.01 | 67.5 | |
| InternVideo2.5-8BParams=8B, Training Scale=16M clips2026.01 | 65.4 | |
| GPTQModel=Qwen2.5-VL-72B-Instruct, Weight bits=3 bit, Memory=28.61 GB, Evaluation protocol=zero-shot2026.07 | 65.11 | |
| SAB-LVLMModel=Qwen2.5-VL-72B-Instruct, Weight bits=1.07 bit, Memory=23.23 GB, Evaluation protocol=zero-shot2026.07 | 64.93 | |
| HINT InternVL3-8BNumber of sampled frames=32, Backbone=InternVL3-8B2026.03 | 64.6 | |
| InternVL3-8BNumber of sampled frames=322026.03 | 64.2 | |
| LLaVA-Video-7BParams=7B, Training Scale=1.3M videos2026.01 | 63.3 | |
| Qwen2-VL-2B + GRPOParams=2B, Training Scale=5.3K QA, Zero-shot evaluation=true2026.01 | 62.11 | |
| Full PrecisionModel=Qwen2.5-VL-7B-Instruct, Weight bits=16 bit, Memory=15.45 GB, Evaluation protocol=zero-shot2026.07 | 61.96 | |
| GPTQModel=Qwen2.5-VL-32B-Instruct, Weight bits=3 bit, Memory=12.71 GB, Evaluation protocol=zero-shot2026.07 | 61.67 | |
| ARB-LLMModel=Qwen2.5-VL-72B-Instruct, Weight bits=1.19 bit, Memory=23.23 GB, Evaluation protocol=zero-shot2026.07 | 61.15 | |
| GPTQModel=Qwen2.5-VL-7B-Instruct, Weight bits=3 bit, Memory=2.66 GB, Evaluation protocol=zero-shot2026.07 | 59.93 | |
| GPT-4VParams=~1.8T, Training Scale=~10T tokens, Zero-shot evaluation=true2026.01 | 59.9 | |
| SAB-LVLMModel=Qwen2.5-VL-32B-Instruct, Weight bits=1.07 bit, Memory=10.28 GB, Evaluation protocol=zero-shot2026.07 | 59.15 | |
| BiLLMModel=Qwen2.5-VL-72B-Instruct, Weight bits=1.08 bit, Memory=24.23 GB, Evaluation protocol=zero-shot2026.07 | 58.41 | |
| ARB-LLMModel=Qwen2.5-VL-32B-Instruct, Weight bits=1.16 bit, Memory=10.28 GB, Evaluation protocol=zero-shot2026.07 | 57.7 | |
| PB-LLMModel=Qwen2.5-VL-72B-Instruct, Weight bits=1.70 bit, Memory=22.99 GB, Evaluation protocol=zero-shot2026.07 | 54.37 | |
| Qwen2-VL-2B (baseline)Params=2B, Training Scale=1.2T tokens2026.01 | 50.1 | |
| SAB-LVLMModel=Qwen2.5-VL-7B-Instruct, Weight bits=1.07 bit, Memory=2.14 GB, Evaluation protocol=zero-shot2026.07 | 48.11 | |
| PB-LLMModel=Qwen2.5-VL-32B-Instruct, Weight bits=1.70 bit, Memory=10.24 GB, Evaluation protocol=zero-shot2026.07 | 47.89 | |
| ARB-LLMModel=Qwen2.5-VL-7B-Instruct, Weight bits=1.07 bit, Memory=2.14 GB, Evaluation protocol=zero-shot2026.07 | 47.14 | |
| BiLLMModel=Qwen2.5-VL-32B-Instruct, Weight bits=1.08 bit, Memory=10.73 GB, Evaluation protocol=zero-shot2026.07 | 47.04 | |
| Video-LLaVA-7BParams=7B, Training Scale=760K videos2026.01 | 45.3 | |
| BiLLMModel=Qwen2.5-VL-7B-Instruct, Weight bits=1.08 bit, Memory=2.24 GB, Evaluation protocol=zero-shot2026.07 | 26.56 | |
| PB-LLMModel=Qwen2.5-VL-7B-Instruct, Weight bits=1.17 bit, Memory=2.15 GB, Evaluation protocol=zero-shot2026.07 | 9.89 |