Multi-video understanding on MVX-Bench
52.7Average ScoreSAMA
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SAMABackbone=Qwen2.5-VL-7B, Reasoning Module=GPT-5.22026.03 | 52.7 | 41.2 | 53.3 | 78 | 64.6 | 57.3 | 54 | 78.8 | 32 | 56.7 | 49.3 | 30 | |
| GPT-5.22026.03 | 50.9 | 32 | 68 | 79.3 | 73.3 | 58 | 57.5 | 81.2 | 24 | 52.7 | 22.4 | 28.7 | |
| GPT-4o2026.03 | 47.7 | 26 | 49 | 71.3 | 70.7 | 40 | 70.8 | 76.8 | 23.3 | 52 | 39.3 | 26.7 | |
| InternVL3.5-38BModel Size=≥ 14B2026.03 | 47.2 | 28.7 | 45 | 71.3 | 53.3 | 58 | 53.8 | 82.6 | 22.7 | 46 | 50.4 | 28.7 | |
| Qwen2.5-VL-32BModel Size=≥ 14B2026.03 | 46.3 | 20.7 | 52 | 72 | 50.7 | 56.7 | 58.5 | 78.3 | 19.3 | 51.3 | 45.3 | 27.3 | |
| Gemma-3-27B-ITModel Size=≥ 14B2026.03 | 44.7 | 28 | 46 | 72.7 | 56 | 56.7 | 33 | 82.6 | 16.7 | 52.7 | 35.9 | 27.3 | |
| InternVL3.5-8BModel Size=7B ≤ Size < 14B2026.03 | 44.7 | 30.7 | 49 | 68.7 | 49.3 | 60 | 26.4 | 81.2 | 21.3 | 51.3 | 43.6 | 26 | |
| InternVL3.5-4BModel Size=< 7B2026.03 | 44.3 | 26.7 | 55 | 64.7 | 34.7 | 57.3 | 48.1 | 79.7 | 25.3 | 53.3 | 44.4 | 22 | |
| InternVL3.5-14BModel Size=≥ 14B2026.03 | 43.8 | 26.7 | 51 | 68 | 44 | 62 | 33 | 79.7 | 22.7 | 46 | 40.2 | 26 | |
| Gemma-3-12B-ITModel Size=7B ≤ Size < 14B2026.03 | 41.5 | 25.3 | 43 | 65.3 | 49.3 | 60 | 16 | 71 | 23.3 | 52 | 36.3 | 22 | |
| Gemma-3-4B-ITModel Size=< 7B2026.03 | 40.6 | 28.7 | 49 | 59.3 | 30.7 | 54 | 16 | 72.5 | 26.7 | 56 | 42.7 | 24 | |
| Qwen2.5-VL-3BModel Size=< 7B2026.03 | 38.1 | 29.3 | 55 | 45.3 | 27.3 | 56 | 34.9 | 63.8 | 16 | 47.3 | 41 | 22.7 | |
| Pixtral-12BModel Size=7B ≤ Size < 14B2026.03 | 35.4 | 23.3 | 43 | 27.3 | 33.3 | 58 | 27.4 | 36.2 | 30.7 | 46.7 | 39.3 | 25.3 | |
| Phi-4-MultimodalModel Size=< 7B2026.03 | 34.4 | 21.3 | 55 | 28.7 | 23.3 | 55.3 | 34.9 | 21.7 | 27.3 | 48 | 42.6 | 22.7 | |
| Idefics3-8B-Llama3Model Size=7B ≤ Size < 14B2026.03 | 33.4 | 25.3 | 51 | 31.3 | 24.7 | 54.7 | 25.5 | 27.5 | 18.7 | 50.7 | 35 | 23.3 | |
| Random2026.03 | 33.2 | 25 | 50 | 25 | 25 | 50 | 25 | 25 | 25 | 50 | 40.5 | 25 | |
| Bee-8BModel Size=7B ≤ Size < 14B2026.03 | 33.2 | 26 | 55 | 28.7 | 23.3 | 52.7 | 34.9 | 24.6 | 26.7 | 32.7 | 45.1 | 22 | |
| LLaVA-OneVision-7BModel Size=7B ≤ Size < 14B2026.03 | 32.7 | 26 | 47 | 29.3 | 26 | 54 | 22.6 | 23.2 | 18 | 52 | 40.5 | 19.3 | |
| Qwen2.5-VL-7BModel Size=7B ≤ Size < 14B2026.03 | 32.6 | 22 | 59 | 24.7 | 21.3 | 58.7 | 21.7 | 23.2 | 22 | 44.7 | 44.4 | 20 | |
| MiniCPM-V-2Model Size=7B ≤ Size < 14B2026.03 | 32 | 28.7 | 41 | 24 | 23.3 | 51.3 | 22.6 | 15.9 | 28 | 49.3 | 38.5 | 22 | |
| Phi-3.5-VisionModel Size=< 7B2026.03 | 22.8 | 22.2 | 13 | 28.1 | 8.8 | 58 | 19.8 | 12.5 | 18.8 | 0 | 9.1 | 25 |