Video Understanding on VideoMME w/ sub (Accuracy)
81.3AccuracyGemini 1.5 Pro
Evaluation Results
| Method | Links | |
|---|---|---|
| Gemini 1.5 ProSetting=best-setting numbers from official reports2026.05 | 81.3 | |
| GPT-4oSetting=best-setting numbers from official reports2026.05 | 77.2 | |
| ParaVT-8BSetting=tool-augmented setting (<think>→<tool_call>→<answer>)2026.05 | 69.4 | |
| Qwen3-VL-8BSetting=tool-augmented setting (<think>→<tool_call>→<answer>)2026.05 | 68.4 | |
| Time-R1-7BSetting=reasoning-enhanced setting (<think>→<answer>)2026.05 | 66.2 | |
| Video-R1-7BSetting=reasoning-enhanced setting (<think>→<answer>)2026.05 | 66 | |
| LongVT-RFT-7BSetting=tool-augmented setting (<think>→<tool_call>→<answer>)2026.05 | 66 | |
| VideoRFT-7BSetting=reasoning-enhanced setting (<think>→<answer>)2026.05 | 65.6 | |
| Video-Thinker-7BSetting=reasoning-enhanced setting (<think>→<answer>)2026.05 | 65.3 | |
| ReWatch-R1-7BSetting=reasoning-enhanced setting (<think>→<answer>)2026.05 | 65 | |
| Qwen2.5-VL-7BSetting=direct-answer setting (native single-pass prompt)2026.05 | 64.5 | |
| Conan-7BSetting=tool-augmented setting (<think>→<tool_call>→<answer>)2026.05 | 62.8 | |
| VideoChat-R1-7BSetting=reasoning-enhanced setting (<think>→<answer>)2026.05 | 58.2 | |
| SAGE-7BSetting=tool-augmented setting (<think>→<tool_call>→<answer>)2026.05 | 52.4 | |
| VideoZoomer-7BSetting=tool-augmented setting (<think>→<tool_call>→<answer>)2026.05 | 48.3 |