Video Reasoning on Video-Holmes (Multi-Metric)
56.1SRVideoSeek
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| VideoSeek#Frames=42.72026.03 | 56.1 | 43.8 | 45 | 54.5 | 46.6 | 43.3 | 41.8 | 47.3 | |
| Gemini 1.5 Pro#Frames=185.1†2026.03 | 52.1 | 48.2 | 34.4 | 26 | 39.2 | 46.4 | 38.9 | 41.2 | |
| GPT-4o#Frames=322026.03 | 50 | 49.6 | 38.8 | 30 | 44 | 39.2 | 37 | 42 | |
| GPT-4oFrame=32, Zero-shot=true2026.03 | 50 | 49.6 | 38.8 | 30 | 44 | 39.2 | 37 | 42 | |
| EVA-GRPOFrame=36.8*, Zero-shot=true2026.03 | 49.3 | 39.5 | 30.4 | 44.5 | 27.1 | 37.6 | 35.2 | 37.2 | |
| Video-R1#Frames=322026.03 | 48.6 | 41.7 | 28.9 | 34.5 | 31 | 33.5 | 35.9 | 36.5 | |
| Video-R1Frame=32, Zero-shot=true2026.03 | 48.6 | 41.7 | 28.9 | 34.5 | 31 | 33.6 | 35.6 | 36.5 | |
| EVA-KTOFrame=5.8*, Zero-shot=true2026.03 | 48.6 | 36.2 | 22.7 | 39.5 | 22.9 | 32 | 31.1 | 32.9 | |
| GPT-5 (Base)#Frames=3842026.03 | 47.2 | 43.4 | 40.6 | 53.5 | 46.3 | 38.1 | 39.6 | 44.1 | |
| Gemini 2.5 Pro#Frames=185.1†2026.03 | 46.6 | 49.3 | 46.9 | 53 | 40.1 | 44.3 | 37.4 | 45 | |
| EVA-SFTFrame=11.5*, Zero-shot=true2026.03 | 44.5 | 33.7 | 26.4 | 39.5 | 23.2 | 31.9 | 32.2 | 32.6 | |
| Gemini 2.0 Flash Thinking#Frames=185.1†2026.03 | 43.4 | 46.9 | 43.1 | 51 | 37.9 | 43.6 | 39.3 | 43.1 | |
| Qwen2.5-VL-32B#Frames=322026.03 | 43.2 | 44.2 | 31.5 | 51 | 36.4 | 31.4 | 32.2 | 38.4 | |
| SEED-Bench-R1#Frames=322026.03 | 42.8 | 35.1 | 25.6 | 40.5 | 29.2 | 29.9 | 32.6 | 33.5 | |
| SEED-Bench-R1Frame=32, Zero-shot=true2026.03 | 42.8 | 35.1 | 25.6 | 40.5 | 29.2 | 29.9 | 32.6 | 33.5 | |
| VideoChat-R1#Frames=322026.03 | 42.1 | 38.8 | 24.5 | 39.5 | 29.5 | 27.8 | 29.3 | 33 | |
| VideoChat-R1Frame=32, Zero-shot=true2026.03 | 42.1 | 38.8 | 24.5 | 39.5 | 29.5 | 27.8 | 29.3 | 33 | |
| Gemini 2.0 Flash#Frames=185.1†2026.03 | 41.8 | 33.7 | 23.1 | 20.5 | 30.1 | 26.8 | 33.7 | 30.6 | |
| Gemini-2.0-FlashZero-shot=true2026.03 | 41.8 | 33.7 | 23.1 | 20.5 | 30.1 | 26.8 | 33.7 | 30.6 | |
| Qwen2.5-VL-7BFrame=32, Zero-shot=true2026.03 | 38.4 | 34.8 | 17.6 | 30 | 27.1 | 18.6 | 25.2 | 27.8 | |
| InternVL3-8BFrame=32, Zero-shot=true2026.03 | 29.5 | 40.7 | 37.9 | 35.1 | 24.6 | 38.9 | 24.1 | 32.3 | |
| InternVL2.5-8BFrame=32, Zero-shot=true2026.03 | 28 | 32.2 | 21.5 | 7.7 | 25.7 | 23.8 | 22.6 | 23.8 |