MMMU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMMU Health & Medicine (test)
74.5Accuracy
39
MMMU (offline)
66.91Accuracy
38
MMMU
56.8Accuracy
38
MMMU (val)
81.8Accuracy
38
MMMU (val)
69.1Accuracy
38
MMMU
57.6Score
36
MMMU-Pro
85.6Accuracy
36
MMMU
75.1Accuracy
35
MMMU
49.1Accuracy
34
MMMU
54.1Accuracy
34
MMMU Pro (Overall)
53.3Score
34
MMMU-Pro
58.4Accuracy
33
MMMU
61.3Accuracy
33
MMMU (val)
69.1Accuracy
33
MMMU-Pro
14.4ECE
32
MMMU in-scope (test)
37Math Score
32
MMMU Pro
56.9Accuracy
31
MMMU Pro
50.7Accuracy
30
MMMU Med
78Accuracy
29
MMMU-Pro
39.42Avg@8
29
Video-MMMU (test)
0.612Accuracy
29
Video-MMMU
87.6Accuracy
29
MMMU (val)
72.6Score
28
MMMU Pro Vision
43.87Accuracy
27
MMMU
57.1MMMU Score
27