MMStar
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMStar
76.6CP Score
14
MMStar 1.0 (test)
63.63Coarse Score
14
MMStar
68.2Score
14
MMStar 2024b
65.3Accuracy
14
MMStar
69.8Accuracy
13
MMStar (val)
70.8Accuracy
13
MMStar
75Accuracy
12
MMStar (test)
60.4Score
12
MMStar (5% Forget Set)
49.27Average Score
12
MMStar
67.1Accuracy (ACC%)
11
MMStar 11 (test)
71.4Average Score
11
MMStar
68.8Score
11
MMStar (test)
69.8MMStar Score
11
MMStar (test)
72.3Accuracy
11
MMStar
71.1MMStar Reasoning Score
10
MMStar 80% (test)
60.08Accuracy
10
MMStar-V
68.4Accuracy
10
MMStar cleaned
77.59Score
10
MMStar fine
62.81Accuracy
9
MMStar
60.3Pass@1 Score
9
MMStar
60.7Accuracy
9
MMStar
67.9MMStar Score
8
MMStar (test)
71.3Overall Accuracy
8
MMStar
83.1Pass@1
7
MMStar
66.4Score
7