VideoMME
Benchmarks
TaskDatasetSOTA resultTrendResults
VideoMME (test)
64.4Overall Score
7
VideoMME Sub (test)
74Score
7
VideoMME (test)
69.3Score
7
VideoMME without subtitles Long
67.4Accuracy
7
VideoMME 64 v2
19.9Score
6
VideoMME long subset w/ sub
81.2Accuracy
6
VideoMME medium
73.89Accuracy
6
VideoMME Medium
73.4Accuracy
6
VideoMME Short
0.718Accuracy
6
VideoMME short, medium, long, overall
73Accuracy (Short Segment)
5
VideoMME with subtitles
88.4Accuracy
5
VideoMME
71.9Accuracy
5
VideoMME 32K context slice
70Score
4
VideoMME 16K context slice
66Score
4
VideoMME w/ audio
89Score
3
VideoMME original (test)
55.9Temporal Score
3
VideoMME 1,000 QA pairs sample
68.3Accuracy (%)
2
VideoMME Long
41.6Accuracy
2
VideoMME 8f (holdout)
1.113Gain
1