Video-MME
Benchmarks
TaskDatasetSOTA resultTrendResults
Video-MME
65.6Accuracy (w/o subtitles)
9
Video-MME no subtitles v2 (test)
14.91Non-Linear Accuracy
9
Video-MME
62.6Performance (w/o Context)
9
Video-MME Short
74.44Accuracy
9
Video-MME
75Score
8
Video-MME
70.2Score
7
Video-MME Long split
35.9Overall Score
6
Video-MME v2
9.4Non-Linear Score
6
Video-MME (test)
40.19Accuracy
6
MT-Video-MME
44.3Accuracy 1 (Acc1)
5
Video-MME short and medium videos
50.1Temporal Accuracy
4
Video-MME
69.7Accuracy
4
Video-MME
62.81Video-MME Score
4
Video-MME Med.
61.45Accuracy
4
Video-MME (VMME) 10-hour variant (no subtitles) 1.0 (test)
69.1Accuracy
4
Video-MME
67.6Accuracy (with subtitle)
4
Video-MME
63.3Accuracy (No Subtitles)
4
Video-MME v2
44.4Accuracy
3
Video-MME v2
35.3ACC (64 frames)
3
Video-MME Short (w/ audio)
85.5Overall Score
3
Video-MME medium
59.9Accuracy
3
Video-MME
60.7Accuracy
3
Video-MME Long Videos
44.6Clip Coherence
3
Video-MME
39.85Acc
3
Video-MME Open-Ended
3Overall Score
2