Video Understanding on Video-Holmes (Accuracy)
58.3AccuracyAdaTooler-V-7B
Evaluation Results
| Method | Links | |
|---|---|---|
| AdaTooler-V-7BFrames=1282025.12 | 58.3 | |
| AdaTooler-V-7BFrames=642025.12 | 56.4 | |
| AdaTooler-V-7BFrames=322025.12 | 55.6 | |
| Gemini 1.5 proModel Category=Proprietary Models, Venue=-2026.03 | 45.7 | |
| Video-Thinker2025.12 | 43.2 | |
| GPT-4o2025.12 | 42 | |
| GPT-4oModel Category=Proprietary Models, Venue=-2026.03 | 42 | |
| VST-7BModel Category=Open-source Online Models, Venue=-2026.03 | 41.9 | |
| Gemini 1.5 Pro2025.12 | 41.2 | |
| InternVL3-8B (MERIT)Model=InternVL3-8B, Variant=MERIT2026.04 | 37.2 | |
| InternVL3-8B (RANDOM-K)Model=InternVL3-8B, Variant=RANDOM-K2026.04 | 36.7 | |
| Video-R12025.12 | 36.5 | |
| Video-R1-7BModel Category=Open-source Offline Models, Venue=NeurIPS’252026.03 | 36.5 | |
| InternVL3-8B (BASE)Model=InternVL3-8B, Variant=BASE2026.04 | 36.5 | |
| InternVL3-8B (ALL-LAYER)Model=InternVL3-8B, Variant=ALL-LAYER2026.04 | 36.2 | |
| Qwen2.5-VL-7B-InstructFrames=1282025.12 | 33.5 | |
| VideoChat-R12025.12 | 33 | |
| Qwen3-VL-4B (MERIT)Model=Qwen3-VL-4B, Variant=MERIT2026.04 | 32.9 | |
| Qwen3-VL-4B (ALL-LAYER)Model=Qwen3-VL-4B, Variant=ALL-LAYER2026.04 | 31.9 | |
| Qwen3-VL-4B (BASE)Model=Qwen3-VL-4B, Variant=BASE2026.04 | 31.5 | |
| Qwen3-VL-4B (RANDOM-K)Model=Qwen3-VL-4B, Variant=RANDOM-K2026.04 | 31 | |
| Qwen2.5-VL-7B-InstructFrames=642025.12 | 29.9 | |
| Qwen2.5-VL-7B-InstructFrames=322025.12 | 27.8 | |
| LongVA-7B (MERIT)Model=LongVA-7B, Variant=MERIT2026.04 | 18.9 | |
| LongVA-7B (ALL-LAYER)Model=LongVA-7B, Variant=ALL-LAYER2026.04 | 15.4 | |
| LongVA-7B (BASE)Model=LongVA-7B, Variant=BASE2026.04 | 15.2 | |
| LongVA-7B (RANDOM-K)Model=LongVA-7B, Variant=RANDOM-K2026.04 | 14.5 |