MMMU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMMU-Pro vision
45.84Accuracy
19
MMMU-Pro
51.8Pass@1
19
MMMU
64.4pass@1
19
MMMU
70Accuracy
19
MMMU (val)
65.34MMMU Score
18
MMMU Pro Vision
51.9Accuracy
18
MMMU
59.2Macro F1
18
MMMU-Pro
70.64Pass@1
18
MMMU (val)
53.1Accuracy
17
MMMUPro
43.47Pass@1
17
MMMU
67.5Accuracy
17
MMMU Reasoning
66.67Accuracy
17
MMMU (val)
70.7Accuracy
17
MMMU
53Accuracy
16
MMMU pt-PT
53.7Accuracy
16
MMMU Video
84.6Accuracy
16
MMMU-Pro
47.1Accuracy@8
16
MMMU-Pro (Standard)
41.5Score
16
MMMU
37.33MMMU Score
16
MMMU
41.1MMMU Score
16
MMMU
60.9Accuracy
15
MMMU (val)
59.9Score
15
MMMU
73.77Accuracy
15
MMMU
78.1Accuracy
15
MMMU ECG
48.5Accuracy
14