List your company

ResearchFormerly WizWand

MME

Benchmarks

TaskDatasetSOTA resultTrendResults

Hallucination Evaluation

MME cognition-related 57 (test)

671.6Accuracy

15

Perception and Cognition Evaluation

MME Perception and Cognition v1.0

1,567.4MME^P Score

15

Hallucination Evaluation

MME Hallucination 9

200Existence

15

Adversarial Attack

MME Untargeted

83.49ASR

15

Adversarial Attack

MME Targeted

59.81ASR

15

MME total perception score

1,711Total Perception Score

15

Multimodal Model Evaluation

MME

1,734.6Perception Score

14

Multimodal Evaluation

MME

1,534.8MME Score

14

Utility Prediction Routing

MME

97.79OpenAI Score

14

Multimodal Evaluation

MME

1,864Perception Score

14

Visual Cognition Hallucination Evaluation

MME Cognition

357.86Cognition Score

14

Visual Perception Hallucination Evaluation

MME Perception

195Existence Fidelity

14

MME

0.729Score

14

Multimodal Evaluation

MME-P

1,687MME-P Score

14

Multimodal Model Evaluation

MME

2,303.4MME Score

13

Multi-modal Perception

MME Perception

1,567.4MME^P Score

13

Multimodal Evaluation

MME

1,851MME Score

13

Multimodal Perception

MME-P (test)

1,111.1MME-P Score

13

Multimodal Reasoning

MME

2,393Sum Score

13

Multimodal Evaluation

MME (test)

1,376.2Perception Score

13

Multi-modal Evaluation

MME

1,818P+C Score

13

Visual Understanding

MME RealWorld

72.7Pass@1 Exact Match

13

Multimodal Perception

MME-RW-cn

72.62Accuracy

13

Multimodal Perception

MME-RW-en

74.86Accuracy

13

General Visual Question Answering

MME 2023

2,339Total Score

13