MMMU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMMU Pro
50.6Accuracy
11
MMMU
57.73Accuracy
11
MMMU
54.67Strict Accuracy
11
MMMU
64.3Accuracy (77.8% reduction ratio)
11
MMMU (val)
71.33Pass@1 Accuracy
11
MMMU
3.74Coherence
10
MMMU Video
48.7Score
10
MMMU Pro
36.63Avg@8 Accuracy
10
MMMU
47.6Accuracy
9
MMMU-Pro-V
26.3Accuracy
9
MMMU
66.7AUROC
9
MMMU Video
83.6Delta Knowledge
9
MMMU (dev)
25.33Score
9
MMMU Pros
75Accuracy
9
MMMU out-of-scope (test)
0.18Refusal Rate
9
MMMU
37Overall Score
9
MMMU
81Detection Accuracy
8
MMMU-Pro
64.81Pass@5
8
MMMU
80.11Pass@5 Score
8
MMMU-P
58.78Accuracy
8
MMMU-Pro overall (test)
28.44Score
8
MMMU (dev)
64.2Score
8
MMMU
55.9Accuracy
8
MMMU
55.7Accuracy
8
MMMU Pro-V-CoT
24.3Accuracy
7