Story Video Question Answering on StoryVideoQA (full set)
55.47P-C AccuracyVideoLLaMA3
Evaluation results
| Method | Links | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| VideoLLaMA3Venue=ArXiv’25, Method Type=MLLMs, Zero-shot=true2026.06 | 55.47 | 71 | 62.85 | 74.06 | 68.88 | 74.21 | 78.99 | 85.56 | 83.91 | 87.03 | 84.22 | 84.6 | 83.63 | 85.53 | 76.32 | |
| ViLAMPVenue=ICML’25, Method Type=MLLMs, Zero-shot=true2026.06 | 54.19 | 66.26 | 58.04 | 70.7 | 67.24 | 71.63 | 76.03 | 84.6 | 82.83 | 84.91 | 83.04 | 83.07 | 82.22 | 83.44 | 73.97 | |
| Video-XLVenue=CVPR’25, Method Type=MLLMs, Zero-shot=true2026.06 | 46.75 | 58.94 | 51.35 | 60.91 | 57.1 | 61.75 | 66.67 | 75.09 | 73.21 | 75.98 | 73.08 | 73.51 | 71.71 | 73.63 | 64.88 | |
| VideoLLaMA2Venue=ArXiv’24, Method Type=MLLMs, Zero-shot=true2026.06 | 44.77 | 47.03 | 52.26 | 58.76 | 58.96 | 62.24 | 71.17 | 79.31 | 77.87 | 81.4 | 79.69 | 79.74 | 78.32 | 80.22 | 66.68 | |
| MA-LMMVenue=CVPR’24, Method Type=MLLMs, Zero-shot=true2026.06 | 40.95 | 45.68 | 48.06 | 54.26 | 55.03 | 58.01 | 64.45 | 72.34 | 71.43 | 74.49 | 71.59 | 72.35 | 71.78 | 74.1 | 61.32 | |
| VideoChat2Venue=CVPR’24, Method Type=MLLMs, Zero-shot=true2026.06 | 35.53 | 39.15 | 45.04 | 49.53 | 50.58 | 52.91 | 62.95 | 65.67 | 65.03 | 69.59 | 66.41 | 66.91 | 65.98 | 69.8 | 56.37 | |
| SeViLAVenue=CVPR’23, Method Type=MLLMs, Zero-shot=true, Backbone=FlanT5-XL 3B2026.06 | 30.32 | 27.38 | 35.44 | 24.31 | 30.74 | 26.65 | 22.05 | 20.78 | 21.08 | 22.61 | 19.47 | 21.69 | 21.21 | 19.18 | 24.89 | |
| Chat-UniViVenue=CVPR’24, Method Type=MLLMs, Zero-shot=true2026.06 | 29.35 | 41 | 25.37 | 37.98 | 28.89 | 38.94 | 31.31 | 28.14 | 32.36 | 19.55 | 28.18 | 19.88 | 29.47 | 22.92 | 30.03 | |
| SINGULARITYVenue=ACL’23, Method Type=VLMs, Zero-shot=true2026.06 | 21.44 | 20.59 | 20.2 | 20.96 | 20.59 | 20.31 | 19.85 | 20.53 | 20.84 | 20.37 | 20.61 | 19.92 | 20.24 | 19.9 | 20.48 | |
| TimeChatVenue=CVPR’24, Method Type=MLLMs, Zero-shot=true2026.06 | 20.97 | 17.47 | 29.83 | 26.28 | 30.84 | 28.24 | 30.07 | 44.24 | 44.13 | 47.08 | 41.19 | 41.69 | 39.93 | 37.72 | 33.49 | |
| VIOLETv2Venue=CVPR’23, Method Type=VLMs, Zero-shot=true2026.06 | 19.27 | 16.32 | 18.18 | 16.86 | 20.32 | 18.45 | 18.3 | 12.03 | 12.69 | 14.15 | 12.54 | 14.7 | 14.68 | 14.09 | 16.06 | |
| Vid-TLDRVenue=CVPR’24, Method Type=VLMs, Zero-shot=true2026.06 | 19.16 | 19.16 | 21.07 | 21.71 | 22.42 | 21.94 | 23.53 | 23.11 | 24.15 | 23.27 | 22.89 | 23.16 | 23.4 | 24.31 | 22.24 | |
| Video-ChatGPTVenue=ACL’24, Method Type=MLLMs, Zero-shot=true2026.06 | 7.41 | 18.16 | 4.34 | 16.94 | 9.52 | 17.05 | 14 | 31.12 | 30.26 | 24.91 | 29.19 | 24.78 | 27.51 | 23.47 | 19.3 |