Multi-episode Video Understanding on SeriesBench
98.2VS ScoreHuman
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Human2025.04 | 98.2 | 94.4 | 94.6 | 97.2 | 92.6 | 95.8 | — | — | — | |
| GPT-4oSize=N/A, PC-DCoT=true2025.04 | 78.6 | 76.1 | 73.8 | 82.1 | 61.7 | 76.2 | 12.61 | 28.29 | 72.75 | |
| InternVL2Size=7B, PC-DCoT=true2025.04 | 76.5 | 71.4 | 67.3 | 81.1 | 66.7 | 73.3 | 8.26 | 20.71 | 67.85 | |
| MiniCPM-V 2.6Size=8B, PC-DCoT=true2025.04 | 76.3 | 68.2 | 66.5 | 82.5 | 60 | 72 | 14.16 | 33.64 | 72.98 | |
| Qwen2-VLSize=7B, PC-DCoT=true2025.04 | 75.2 | 73.5 | 69.7 | 80.9 | 67.4 | 73.9 | 8.93 | 23.41 | 68.08 | |
| GPT-4oSize=N/A, Frames=50, PC-DCoT=false2025.04 | 55.8 | 62.8 | 60.6 | 79.9 | 59.6 | 62.8 | 9.61 | 25.1 | 68.94 | |
| Qwen2-VLSize=7B, Frames=64, PC-DCoT=false2025.04 | 55.7 | 57.5 | 58.6 | 75.3 | 59.6 | 60.3 | 11.41 | 27.97 | 70.71 | |
| LLaVA-VideoSize=7B, Frames=642025.04 | 54.9 | 56.8 | 57.4 | 71.2 | 51.9 | 58.3 | 7.27 | 19.06 | 67.95 | |
| MiniCPM-V 2.6Size=8B, Frames=64, PC-DCoT=false2025.04 | 53.3 | 57.3 | 57.7 | 76.6 | 55.1 | 59.1 | 8.57 | 30.06 | 68.66 | |
| AriaSize=8x3.5B, Frames=1282025.04 | 52.4 | 56.6 | 51.8 | 75.6 | 50 | 56.8 | 10 | 26.08 | 70.35 | |
| InternVL2Size=7B, Frames=32, PC-DCoT=false2025.04 | 52 | 56.8 | 57.7 | 78.6 | 57.7 | 59.2 | 8.43 | 21.22 | 68.4 | |
| LLaVA-OneVisionSize=7B, Frames=322025.04 | 51.5 | 54 | 56.2 | 70.6 | 59.6 | 56.9 | 7.3 | 20.2 | 67.82 | |
| VideoLLaMA2.1-AVSize=7B, Frames=32, Audio=false2025.04 | 47.4 | 52.2 | 53.3 | 66.6 | 49.4 | 53.1 | 7.61 | 23.34 | 67.53 | |
| VITASize=8x7B, Frames=322025.04 | 46.9 | 46 | 53.3 | 68.9 | 48.1 | 51.8 | 9.32 | 27.08 | 68.93 | |
| GPT-4o-miniSize=N/A, Frames=502025.04 | 46.7 | 42.7 | 47.9 | 70.9 | 44.9 | 49.8 | 9.99 | 29.3 | 68.76 | |
| VideoLLaMA2.1-AVSize=7B, Frames=32, Audio=true2025.04 | 45.3 | 52.2 | 51.3 | 66.2 | 46.2 | 51.7 | 7.24 | 22.71 | 67.09 | |
| Frequent Guess2025.04 | 43.6 | 46 | 43.1 | 41.1 | 50.6 | 44.4 | — | — | — | |
| Random Choice2025.04 | 39.3 | 38.2 | 35.5 | 36.5 | 38.8 | 37.7 | — | — | — |