Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

MMStar dataset and benchmarks – Page 4 · SOTA2 Research

Research Datasets

MMStar

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Vision-Language Understanding

MMStar (test)

63.73Accuracy

7

Visual Question Answering

MMStar 2024b

55.1Score

7

Visual Question Answering

MMStar (val)

59.8CP Score

7

Fine-grained Visual Reasoning

MMStar

77.5Score

6

Visual Question Answering

MMStar

68.2Score

6

Multidisciplinary Reasoning

MMStar

75.1Accuracy

6

General Multimodal Capabilities

MMStar

79.4Accuracy

6

Multimodal Vision-Language Evaluation

MMStar 1.0 (test)

69.6Contrastive Prediction Score

6

Multimodal Reasoning

MMStar v1.0 (test)

56.7Accuracy

5

Abstract reasoning

MMStar

63.21Accuracy

5

Multimodal Understanding

MMStar

61.53Overall Score

5

Multi-modal Model Evaluation

MMStar

47.6Noise Score

4

Image Chat / Understanding

MMStar

64.2Accuracy

4

Mathematical multi-modal reasoning

MMStar

62.53pass@1

4

Vision-Language Understanding

MMStar

42.64Accuracy

3

Multi-modal reasoning

MMStar

43.7MMStar Score

3

Visual perception and reasoning

MMStar avg

39.4Average Accuracy

3

General Multimodal Question Answering

MMStar

71.1Accuracy

3

MMStar

55Pass@1

3

Page 4 of 4

Popular tasks

Multimodal Reasoning Multimodal Understanding Visual Question Answering General VQA Perception

Follow for updates

Join SOTA2 Contact