MMEB
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMEB
788.1Classification Score
94
MMEB
76.1Classification Accuracy
79
MMEB v1 (test)
69.1Classification
70
MMEB 1.0 (test)
67.6Classification Accuracy
52
MMEB Image V2
78.2Overall Score
37
MMEB V2 (test)
76.7Image CLS Acc
37
MMEB V2 (test)
69.8Image CLS Hit@1
35
MMEB V1
69.7Classification Accuracy
31
MMEB Visual Document portion v2
89.44ViDoRe V1 Score
31
MMEB
70Classification Score
22
MMEB Full v2 (test)
77.8Overall Average Score
18
MMEB Video V2 (test)
78.4CLS Hit@1
18
MMEB Image v2 (test)
76.1CLS (Hit@1)
18
MMEB v1 (test)
61.2Classification
18
MMEB OOD 1.0
59.1OOD Precision@1
18
MMEB In-Distribution 1.0
71.6MMEB IND Precision@1
18
MMEB Overall 1.0
61.6Classification P@1
18
MMEB Overall
72.6Classification Score
18
MMEB v2
78.2Image Retrieval Score
18
MMEB Video v2
57.8Classification Score (CLS)
17
MMEB V2
70Image CLS Accuracy
16
MMEB Summary v2 (Overall)
79.2Overall Score
15
MMEB VisDoc v2
87.1Overall Score
15
MMEB Video v2
62.9Overall Average Score
15
MMEB Total v2
68.1Overall Score
15