MMMU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMMU (val)
69.1Accuracy
14
MMMU
92.3OpenAI Score
14
MMMU-Pro
55.12Accuracy
14
MMMU-Pro
14.98ECE
14
MMMU-Pro (test)
55Accuracy
14
MMMU standard (test)
74.3MMMU Score
14
MMMU 1.0 (test)
50.6MMMU Score
14
MMMU-Pro
55.9Accuracy (Reasoning on MMMU-Pro)
13
MMMU
68.3Accuracy (MMMU Multi-step Reasoning)
13
MMMU-Pro avg
45.6Accuracy
13
MMMU (dev val)
52.8Pass@1
13
MMMU (val)
52.4MMMU Score
13
MMMU 2024 (val)
52.8Accuracy
13
MMMU Med
78Accuracy
12
MMMU (dev)
88.2Score
12
MMMU
54.2Accuracy
12
MMMU
54.11Accuracy
12
MMMU
54.35Utility (OpenAI)
12
MMMU-Pro 1.0 (test)
44.39Overall Score
12
MMMU
57.6MMMU Score
12
MMMU Video
55.91Accuracy
12
MMMU OOD
69.2ARD
12
MMMU Health & Medicine
0.694Accuracy
12
MMMU
51.4Accuracy
11
MMMU 2024 (val)
58Score
11