# Massive Multi-discipline Multimodal Understanding on MMMU

Source: [Massive Multi-discipline Multimodal Understanding on MMMU](<https://www.sota2.com/research/sota/massive-multi-discipline-multimodal-understanding-on-mmmu>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-19T08:31:19.560Z

Evaluation of model performance on the MMMU benchmark using the full 'Massive Multi-discipline Multimodal Understanding' task definition, reporting standard score and relative performance metrics.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Massive Multi-discipline Multimodal Understanding |
| Dataset (including any stated split) | MMMU |
| Metric | Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 327 of 327 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Accuracy](<https://www.sota2.com/research/sota/massive-multi-discipline-multimodal-understanding-on-mmmu.md?metric=Accuracy>): Higher is better; unit: PERCENT.

- [Score](<https://www.sota2.com/research/sota/massive-multi-discipline-multimodal-understanding-on-mmmu.md?metric=Score>): Higher is better; unit: SCALAR.

- [Relative Performance](<https://www.sota2.com/research/sota/massive-multi-discipline-multimodal-understanding-on-mmmu.md?metric=Relative%20Performance>): Higher is better; unit: PERCENT.

- [Relative Score](<https://www.sota2.com/research/sota/massive-multi-discipline-multimodal-understanding-on-mmmu.md?metric=Relative%20Score>): Higher is better; unit: PERCENT.

- [Relative Accuracy](<https://www.sota2.com/research/sota/massive-multi-discipline-multimodal-understanding-on-mmmu.md?metric=Relative%20Accuracy>): Higher is better; unit: PERCENT.

- [BoN@8 Accuracy](<https://www.sota2.com/research/sota/massive-multi-discipline-multimodal-understanding-on-mmmu.md?metric=BoN%408%20Accuracy>): Higher is better; unit: PERCENT.

- [Performance Delta (BoN@8 vs Pass@1)](<https://www.sota2.com/research/sota/massive-multi-discipline-multimodal-understanding-on-mmmu.md?metric=Performance%20Delta%20(BoN%408%20vs%20Pass%401)>): Higher is better; unit: SCALAR.

## Recorded results

| Method | Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| SPOT-E | 71.5 | Not recorded | 6a34cd17206542251d7c5f2e |
| SPOT-E | 70.4 | Not recorded | 6a34cd17206542251d7c5f2e |
| Gemini-2.5-Flash | 70 | Not recorded | 6a34cd17206542251d7c5f2e |
| GPT-4o | 69.2 | Not recorded | 6a34cd17206542251d7c5f2e |
| DART | 65.5 | Not recorded | 6937ad116cfd488761a4e841 |
| SPOT-E | 65.2 | Not recorded | 6a34cd17206542251d7c5f2e |
| Debate with Consensus | 63.1 | Not recorded | 6937ad116cfd488761a4e841 |
| Debate with Judge | 63.1 | Not recorded | 6937ad116cfd488761a4e841 |
| InternVL3-8B | 62.7 | Not recorded | 6a34cd17206542251d7c5f2e |
| Qwen3-VL-8B-Instruct | 62.4 | Not recorded | 6996994609c33900fcd720a7 |
| Qwen3-VL-8B-Thinking | 62.3 | Not recorded | 6996994609c33900fcd720a7 |
| SAP | 62.3 | Not recorded | 6996994609c33900fcd720a7 |
| SPOT-E | 62 | Not recorded | 6a34cd17206542251d7c5f2e |
| DART | 61.6 | Not recorded | 6937ad116cfd488761a4e841 |
| Qwen 3.5 | 61.22 | Not recorded | 6a02b479dbc836b87d824bfb |
| SPOT-E | 61 | Not recorded | 6a34cd17206542251d7c5f2e |
| Self-Consistency (5-way) | 60.8 | Not recorded | 6937ad116cfd488761a4e841 |
| Phoenix-VL 1.5 Medium | 60.78 | Not recorded | 6a02b479dbc836b87d824bfb |
| Debate with Consensus | 60.5 | Not recorded | 6937ad116cfd488761a4e841 |
| Debate with Judge | 60.3 | Not recorded | 6937ad116cfd488761a4e841 |
| GPT-4o-mini | 60 | Not recorded | 6a34cd17206542251d7c5f2e |
| SPOT-E | 59.5 | Not recorded | 6a34cd17206542251d7c5f2e |
| Claude-3-Opus | 59.4 | Not recorded | 6996994609c33900fcd720a7 |
| CoT | 58.8 | Not recorded | 6937ad116cfd488761a4e841 |
| Gemini-1.5 | 58.5 | Not recorded | 6996994609c33900fcd720a7 |
| SPOT-E | 58.5 | Not recorded | 6a34cd17206542251d7c5f2e |
| Debate with Judge | 58.3 | Not recorded | 6937ad116cfd488761a4e841 |
| Qwen3-VL-8B | 58 | Not recorded | 6a34cd17206542251d7c5f2e |
| Debate with Consensus | 57 | Not recorded | 6937ad116cfd488761a4e841 |
| GPT-4V | 56.8 | Not recorded | 6996994609c33900fcd720a7 |
| Self-Refinement | 56.5 | Not recorded | 6937ad116cfd488761a4e841 |
| InternVL2.5-8B | 56 | Not recorded | 6a34cd17206542251d7c5f2e |
| Qwen2.5-VL-7B | 55 | Not recorded | 6a34cd17206542251d7c5f2e |
| Debate with Judge | 54.8 | Not recorded | 6937ad116cfd488761a4e841 |
| Qwen2-VL | 54.1 | Not recorded | 68fa77981e209131cea743dd |
| ViperGPT | 54 | Not recorded | 6937ad116cfd488761a4e841 |
| Grok-1.5V | 53.6 | Not recorded | 6996994609c33900fcd720a7 |
| Debate with Consensus | 53.2 | Not recorded | 6937ad116cfd488761a4e841 |
| Full Cache | 53.18 | Not recorded | 6923e6a0d6b0dd334292f0f0 |
| H2O | 53.18 | Not recorded | 6923e6a0d6b0dd334292f0f0 |
| SnapKV | 53.18 | Not recorded | 6923e6a0d6b0dd334292f0f0 |
| LOOK-M | 53.18 | Not recorded | 6923e6a0d6b0dd334292f0f0 |
| MEDA | 53.18 | Not recorded | 6923e6a0d6b0dd334292f0f0 |
| FlashCache | 53.18 | Not recorded | 6923e6a0d6b0dd334292f0f0 |
| Claude-3-Sonnet | 53.1 | Not recorded | 6996994609c33900fcd720a7 |
| Self-Consistency (5-way) | 52.8 | Not recorded | 6937ad116cfd488761a4e841 |
| Self-Consistency (5-way) | 52.5 | Not recorded | 6937ad116cfd488761a4e841 |
| FlashCache | 52.27 | Not recorded | 6923e6a0d6b0dd334292f0f0 |
| SnapKV | 52.18 | Not recorded | 6923e6a0d6b0dd334292f0f0 |
| LOOK-M | 52.18 | Not recorded | 6923e6a0d6b0dd334292f0f0 |
| Chameleon | 51.6 | Not recorded | 6937ad116cfd488761a4e841 |
| MEDA | 51.36 | Not recorded | 6923e6a0d6b0dd334292f0f0 |
| H2O | 51.18 | Not recorded | 6923e6a0d6b0dd334292f0f0 |
| Qwen2.5-VL 7B, Dense | 50.78 | Not recorded | 69ba31a70636e5bf4afc0941 |
| CoT | 50.7 | Not recorded | 6937ad116cfd488761a4e841 |
| CoT | 50.4 | Not recorded | 6937ad116cfd488761a4e841 |
| StreamingLLM | 50 | Not recorded | 6923e6a0d6b0dd334292f0f0 |
| StreamingLLM | 50 | Not recorded | 6923e6a0d6b0dd334292f0f0 |
| Gemma3-27B | 49.7 | Not recorded | 6944dc2b423ea42265729d1f |
| LLaVA-OneVision 7B, Dense | 49.22 | Not recorded | 69ba31a70636e5bf4afc0941 |
| GLM-4.5V | 48.5 | Not recorded | 6a02b479dbc836b87d824bfb |
| Self-Refinement | 47.2 | Not recorded | 6937ad116cfd488761a4e841 |
| Self-Refinement | 46.7 | Not recorded | 6937ad116cfd488761a4e841 |
| Phi-3-Vision-4B | 46.1 | Not recorded | 68fa77611e209131cea72d70 |
| Gemma3-4B + AuditDM | 45.2 | Not recorded | 6944dc2b423ea42265729d1f |
| LWM | 44.8 | Not recorded | 68fa77981e209131cea743dd |
| Gemma3-12B | 44.8 | Not recorded | 6944dc2b423ea42265729d1f |
| Full Cache | 43.34 | Not recorded | 69fc1c5e5497b3b670c88334 |
| PSFT | 43.33 | Not recorded | 68fa7ae21e209131cea85a73 |
| Llama 4 Maverick | 43 | Not recorded | 6a02b479dbc836b87d824bfb |
| SMoE (k=2) | 42.67 | Not recorded | 6a337b91206542251d7c5083 |
| SPOT-E | 42.5 | Not recorded | 6a34cd17206542251d7c5f2e |
| RetentiveKV | 42.45 | Not recorded | 69fc1c5e5497b3b670c88334 |
| SmoothSMoE annealed (k=2) | 42.44 | Not recorded | 6a337b91206542251d7c5083 |
| H2O | 42.39 | Not recorded | 69fc1c5e5497b3b670c88334 |
| Meda | 42.36 | Not recorded | 69fc1c5e5497b3b670c88334 |
| SnapKV | 42.33 | Not recorded | 69fc1c5e5497b3b670c88334 |
| Full Cache | 42.33 | Not recorded | 69fc1c5e5497b3b670c88334 |
| SAINT | 42.28 | Not recorded | 69fc1c5e5497b3b670c88334 |
| Qwen2-VL-2B | 42.2 | Not recorded | 68fa77611e209131cea72d70 |

### SPOT-E

Accuracy: 71.5. Result ID: 6a34fe4ffbc373592c325ef6.

[Reporting paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

Recorded evaluation configuration: {"Base Model":"Gemini-2.5-Flash","Source Type":"Closed-Source","Test-time Adaptation":"SPOT-E"}. This record does not assert independent reproduction.

### SPOT-E

Accuracy: 70.4. Result ID: 6a34fe4ffbc373592c325ef2.

[Reporting paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

Recorded evaluation configuration: {"Base Model":"GPT-4o","Source Type":"Closed-Source","Test-time Adaptation":"SPOT-E"}. This record does not assert independent reproduction.

### Gemini-2.5-Flash

Accuracy: 70. Result ID: 6a34fe4ffbc373592c325ef5.

[Reporting paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c311e209131cea8cef5.md>)

Recorded evaluation configuration: {"Base Model":"Gemini-2.5-Flash","Source Type":"Closed-Source","Test-time Adaptation":"None"}. This record does not assert independent reproduction.

### GPT-4o

Accuracy: 69.2. Result ID: 6a34fe4ffbc373592c325ef1.

[Reporting paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75681e209131cea67a98.md>)

Recorded evaluation configuration: {"Base Model":"GPT-4o","Source Type":"Closed-Source","Test-time Adaptation":"None"}. This record does not assert independent reproduction.

### DART

Accuracy: 65.5. Result ID: 6996c8685698665ec41414d7.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

Recorded evaluation configuration: {"Agent":"Q, M, O"}. This record does not assert independent reproduction.

### SPOT-E

Accuracy: 65.2. Result ID: 6a34fe4ffbc373592c325f02.

[Reporting paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

Recorded evaluation configuration: {"Base Model":"InternVL3-8B","Source Type":"Open-Source","Test-time Adaptation":"SPOT-E"}. This record does not assert independent reproduction.

### Debate with Consensus

Accuracy: 63.1. Result ID: 6996c8685698665ec41414d1.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b61e209131cea5e0f5.md>)

Recorded evaluation configuration: {"Agent":"Q, M, O"}. This record does not assert independent reproduction.

### Debate with Judge

Accuracy: 63.1. Result ID: 6996c8685698665ec41414d5.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b61e209131cea5e0f5.md>)

Recorded evaluation configuration: {"Agent":"Q, M, O"}. This record does not assert independent reproduction.

### InternVL3-8B

Accuracy: 62.7. Result ID: 6a34fe4ffbc373592c325f01.

[Reporting paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71de1e209131cea5370d.md>)

[Implementation](<https://github.com/OpenGVLab/InternVL>)

Recorded evaluation configuration: {"Base Model":"InternVL3-8B","Source Type":"Open-Source","Test-time Adaptation":"None"}. This record does not assert independent reproduction.

### Qwen3-VL-8B-Instruct

Accuracy: 62.4. Result ID: 699af6345698665ec4879262.

[Reporting paper](<https://www.sota2.com/research/paper/6996994609c33900fcd720a7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69608b6653fdd5c3617441a2.md>)

[Implementation](<https://github.com/QwenLM/Qwen3-VL-Embedding>)

Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B","Strategy":"Instruct"}. This record does not assert independent reproduction.

### Qwen3-VL-8B-Thinking

Accuracy: 62.3. Result ID: 699af6345698665ec4879263.

[Reporting paper](<https://www.sota2.com/research/paper/6996994609c33900fcd720a7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69608b6653fdd5c3617441a2.md>)

[Implementation](<https://github.com/QwenLM/Qwen3-VL-Embedding>)

Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B","Strategy":"LongCoT (Thinking)"}. This record does not assert independent reproduction.

### SAP

Accuracy: 62.3. Result ID: 699af6345698665ec4879264.

[Reporting paper](<https://www.sota2.com/research/paper/6996994609c33900fcd720a7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6996994609c33900fcd720a7.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B","Strategy":"SAP"}. This record does not assert independent reproduction.

### SPOT-E

Accuracy: 62. Result ID: 6a34fe4ffbc373592c325ef4.

[Reporting paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

Recorded evaluation configuration: {"Base Model":"GPT-4o-mini","Source Type":"Closed-Source","Test-time Adaptation":"SPOT-E"}. This record does not assert independent reproduction.

### DART

Accuracy: 61.6. Result ID: 6996c8685698665ec41414d6.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

Recorded evaluation configuration: {"Agent":"Best Model"}. This record does not assert independent reproduction.

### Qwen 3.5

Accuracy: 61.22. Result ID: 6a03491b47f34ce924e87991.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b479dbc836b87d824bfb.md>)

Recorded evaluation configuration: {"Zero-shot":"true","Parameters":"122B-A10B","Organization":"Alibaba Cloud"}. This record does not assert independent reproduction.

### SPOT-E

Accuracy: 61. Result ID: 6a34fe4ffbc373592c325efa.

[Reporting paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-VL-8B","Source Type":"Open-Source","Test-time Adaptation":"SPOT-E"}. This record does not assert independent reproduction.

### Self-Consistency (5-way)

Accuracy: 60.8. Result ID: 6996c8685698665ec41414ca.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6eb61e209131cea41b33.md>)

Recorded evaluation configuration: {"Agent":"QwenVL"}. This record does not assert independent reproduction.

### Phoenix-VL 1.5 Medium

Accuracy: 60.78. Result ID: 6a03491b47f34ce924e8798e.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b479dbc836b87d824bfb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a02b479dbc836b87d824bfb.md>)

Recorded evaluation configuration: {"Zero-shot":"true","Parameters":"123B","Organization":"HTX"}. This record does not assert independent reproduction.

### Debate with Consensus

Accuracy: 60.5. Result ID: 6996c8685698665ec41414ce.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b61e209131cea5e0f5.md>)

Recorded evaluation configuration: {"Agent":"3×QwenVL"}. This record does not assert independent reproduction.

### Debate with Judge

Accuracy: 60.3. Result ID: 6996c8685698665ec41414d2.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b61e209131cea5e0f5.md>)

Recorded evaluation configuration: {"Agent":"3×QwenVL"}. This record does not assert independent reproduction.

### GPT-4o-mini

Accuracy: 60. Result ID: 6a34fe4ffbc373592c325ef3.

[Reporting paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75681e209131cea67a98.md>)

Recorded evaluation configuration: {"Base Model":"GPT-4o-mini","Source Type":"Closed-Source","Test-time Adaptation":"None"}. This record does not assert independent reproduction.

### SPOT-E

Accuracy: 59.5. Result ID: 6a34fe4ffbc373592c325f00.

[Reporting paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

Recorded evaluation configuration: {"Base Model":"InternVL2.5-8B","Source Type":"Open-Source","Test-time Adaptation":"SPOT-E"}. This record does not assert independent reproduction.

### Claude-3-Opus

Accuracy: 59.4. Result ID: 699af6345698665ec4879268.

[Reporting paper](<https://www.sota2.com/research/paper/6996994609c33900fcd720a7.md>)

Recorded evaluation configuration: {"Table Header":"Claude-O"}. This record does not assert independent reproduction.

### CoT

Accuracy: 58.8. Result ID: 6996c8685698665ec41414c2.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e571e209131cea3f27b.md>)

Recorded evaluation configuration: {"Agent":"QwenVL"}. This record does not assert independent reproduction.

### Gemini-1.5

Accuracy: 58.5. Result ID: 699af6345698665ec4879269.

[Reporting paper](<https://www.sota2.com/research/paper/6996994609c33900fcd720a7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76361e209131cea6be11.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### SPOT-E

Accuracy: 58.5. Result ID: 6a34fe4ffbc373592c325ef8.

[Reporting paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-7B","Source Type":"Open-Source","Test-time Adaptation":"SPOT-E"}. This record does not assert independent reproduction.

### Debate with Judge

Accuracy: 58.3. Result ID: 6996c8685698665ec41414d4.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b61e209131cea5e0f5.md>)

Recorded evaluation configuration: {"Agent":"3×Ovis2"}. This record does not assert independent reproduction.

### Qwen3-VL-8B

Accuracy: 58. Result ID: 6a34fe4ffbc373592c325ef9.

[Reporting paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6927db54d6b0dd3342930f66.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-VL-8B","Source Type":"Open-Source","Test-time Adaptation":"None"}. This record does not assert independent reproduction.

### Debate with Consensus

Accuracy: 57. Result ID: 6996c8685698665ec41414d0.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b61e209131cea5e0f5.md>)

Recorded evaluation configuration: {"Agent":"3×Ovis2"}. This record does not assert independent reproduction.

### GPT-4V

Accuracy: 56.8. Result ID: 699af6345698665ec4879266.

[Reporting paper](<https://www.sota2.com/research/paper/6996994609c33900fcd720a7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa727a1e209131cea56a1e.md>)

[Implementation](<https://github.com/openai/evals>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### Self-Refinement

Accuracy: 56.5. Result ID: 6996c8685698665ec41414c6.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f7f1e209131cea45fc4.md>)

Recorded evaluation configuration: {"Agent":"QwenVL"}. This record does not assert independent reproduction.

### InternVL2.5-8B

Accuracy: 56. Result ID: 6a34fe4ffbc373592c325eff.

[Reporting paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71de1e209131cea5370d.md>)

[Implementation](<https://github.com/OpenGVLab/InternVL>)

Recorded evaluation configuration: {"Base Model":"InternVL2.5-8B","Source Type":"Open-Source","Test-time Adaptation":"None"}. This record does not assert independent reproduction.

### Qwen2.5-VL-7B

Accuracy: 55. Result ID: 6a34fe4ffbc373592c325ef7.

[Reporting paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75151e209131cea65976.md>)

[Implementation](<https://github.com/QwenLM/Qwen2-VL>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-7B","Source Type":"Open-Source","Test-time Adaptation":"None"}. This record does not assert independent reproduction.

### Debate with Judge

Accuracy: 54.8. Result ID: 6996c8685698665ec41414d3.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b61e209131cea5e0f5.md>)

Recorded evaluation configuration: {"Agent":"3×MiniCPM-o"}. This record does not assert independent reproduction.

### Qwen2-VL

Accuracy: 54.1. Result ID: 69954a633badfaffe6395cb5.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77981e209131cea743dd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75151e209131cea65976.md>)

[Implementation](<https://github.com/QwenLM/Qwen2-VL>)

Recorded evaluation configuration: {"LLM Param":"7B","Type":"Und. Only"}. This record does not assert independent reproduction.

### ViperGPT

Accuracy: 54. Result ID: 6996c8685698665ec41414c0.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6ec31e209131cea41d4d.md>)

[Implementation](<https://www.sota2.com/viper.cs.columbia.edu/>)

Recorded evaluation configuration: {"Agent":"Qwen"}. This record does not assert independent reproduction.

### Grok-1.5V

Accuracy: 53.6. Result ID: 699af6345698665ec4879265.

[Reporting paper](<https://www.sota2.com/research/paper/6996994609c33900fcd720a7.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### Debate with Consensus

Accuracy: 53.2. Result ID: 6996c8685698665ec41414cf.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b61e209131cea5e0f5.md>)

Recorded evaluation configuration: {"Agent":"3×MiniCPM-o"}. This record does not assert independent reproduction.

### Full Cache

Accuracy: 53.18. Result ID: 69aa961b6dbb7075b9547548.

[Reporting paper](<https://www.sota2.com/research/paper/6923e6a0d6b0dd334292f0f0.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-VL-7B-Instruct"}. This record does not assert independent reproduction.

### H2O

Accuracy: 53.18. Result ID: 69aa961b6dbb7075b954754b.

[Reporting paper](<https://www.sota2.com/research/paper/6923e6a0d6b0dd334292f0f0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa718c1e209131cea51ead.md>)

[Implementation](<https://github.com/FMInference/H2O>)

Recorded evaluation configuration: {"retention ratio ρ":"0.1","Backbone":"Qwen2.5-VL-7B-Instruct"}. This record does not assert independent reproduction.

### SnapKV

Accuracy: 53.18. Result ID: 69aa961b6dbb7075b954754d.

[Reporting paper](<https://www.sota2.com/research/paper/6923e6a0d6b0dd334292f0f0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b21e209131cea5df89.md>)

[Implementation](<https://github.com/FasterDecoding/SnapKV>)

Recorded evaluation configuration: {"retention ratio ρ":"0.1","Backbone":"Qwen2.5-VL-7B-Instruct"}. This record does not assert independent reproduction.

### LOOK-M

Accuracy: 53.18. Result ID: 69aa961b6dbb7075b954754f.

[Reporting paper](<https://www.sota2.com/research/paper/6923e6a0d6b0dd334292f0f0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73d31e209131cea5e5a8.md>)

Recorded evaluation configuration: {"retention ratio ρ":"0.1","Backbone":"Qwen2.5-VL-7B-Instruct"}. This record does not assert independent reproduction.

### MEDA

Accuracy: 53.18. Result ID: 69aa961b6dbb7075b9547551.

[Reporting paper](<https://www.sota2.com/research/paper/6923e6a0d6b0dd334292f0f0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77611e209131cea72d66.md>)

Recorded evaluation configuration: {"retention ratio ρ":"0.1","Backbone":"Qwen2.5-VL-7B-Instruct"}. This record does not assert independent reproduction.

### FlashCache

Accuracy: 53.18. Result ID: 69aa961b6dbb7075b9547553.

[Reporting paper](<https://www.sota2.com/research/paper/6923e6a0d6b0dd334292f0f0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6923e6a0d6b0dd334292f0f0.md>)

Recorded evaluation configuration: {"retention ratio ρ":"0.1","Backbone":"Qwen2.5-VL-7B-Instruct"}. This record does not assert independent reproduction.

### Claude-3-Sonnet

Accuracy: 53.1. Result ID: 699af6345698665ec4879267.

[Reporting paper](<https://www.sota2.com/research/paper/6996994609c33900fcd720a7.md>)

Recorded evaluation configuration: {"Table Header":"Claude-S"}. This record does not assert independent reproduction.

### Self-Consistency (5-way)

Accuracy: 52.8. Result ID: 6996c8685698665ec41414cc.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6eb61e209131cea41b33.md>)

Recorded evaluation configuration: {"Agent":"Ovis2"}. This record does not assert independent reproduction.

### Self-Consistency (5-way)

Accuracy: 52.5. Result ID: 6996c8685698665ec41414cb.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6eb61e209131cea41b33.md>)

Recorded evaluation configuration: {"Agent":"MiniCPM-o"}. This record does not assert independent reproduction.

### FlashCache

Accuracy: 52.27. Result ID: 69aa961b6dbb7075b9547554.

[Reporting paper](<https://www.sota2.com/research/paper/6923e6a0d6b0dd334292f0f0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6923e6a0d6b0dd334292f0f0.md>)

Recorded evaluation configuration: {"retention ratio ρ":"0.05","Backbone":"Qwen2.5-VL-7B-Instruct"}. This record does not assert independent reproduction.

### SnapKV

Accuracy: 52.18. Result ID: 69aa961b6dbb7075b954754e.

[Reporting paper](<https://www.sota2.com/research/paper/6923e6a0d6b0dd334292f0f0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b21e209131cea5df89.md>)

[Implementation](<https://github.com/FasterDecoding/SnapKV>)

Recorded evaluation configuration: {"retention ratio ρ":"0.05","Backbone":"Qwen2.5-VL-7B-Instruct"}. This record does not assert independent reproduction.

### LOOK-M

Accuracy: 52.18. Result ID: 69aa961b6dbb7075b9547550.

[Reporting paper](<https://www.sota2.com/research/paper/6923e6a0d6b0dd334292f0f0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73d31e209131cea5e5a8.md>)

Recorded evaluation configuration: {"retention ratio ρ":"0.05","Backbone":"Qwen2.5-VL-7B-Instruct"}. This record does not assert independent reproduction.

### Chameleon

Accuracy: 51.6. Result ID: 6996c8685698665ec41414c1.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa710e1e209131cea4eee8.md>)

[Implementation](<https://chameleon-llm.github.io/>)

Recorded evaluation configuration: {"Agent":"Qwen"}. This record does not assert independent reproduction.

### MEDA

Accuracy: 51.36. Result ID: 69aa961b6dbb7075b9547552.

[Reporting paper](<https://www.sota2.com/research/paper/6923e6a0d6b0dd334292f0f0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77611e209131cea72d66.md>)

Recorded evaluation configuration: {"retention ratio ρ":"0.05","Backbone":"Qwen2.5-VL-7B-Instruct"}. This record does not assert independent reproduction.

### H2O

Accuracy: 51.18. Result ID: 69aa961b6dbb7075b954754c.

[Reporting paper](<https://www.sota2.com/research/paper/6923e6a0d6b0dd334292f0f0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa718c1e209131cea51ead.md>)

[Implementation](<https://github.com/FMInference/H2O>)

Recorded evaluation configuration: {"retention ratio ρ":"0.05","Backbone":"Qwen2.5-VL-7B-Instruct"}. This record does not assert independent reproduction.

### Qwen2.5-VL 7B, Dense

Accuracy: 50.78. Result ID: 69ba5b8447f34ce924a40844.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31a70636e5bf4afc0941.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa773a1e209131cea71d3c.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-VL 7B","Sparsity":"Dense"}. This record does not assert independent reproduction.

### CoT

Accuracy: 50.7. Result ID: 6996c8685698665ec41414c4.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e571e209131cea3f27b.md>)

Recorded evaluation configuration: {"Agent":"Ovis2"}. This record does not assert independent reproduction.

### CoT

Accuracy: 50.4. Result ID: 6996c8685698665ec41414c3.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e571e209131cea3f27b.md>)

Recorded evaluation configuration: {"Agent":"MiniCPM-o"}. This record does not assert independent reproduction.

### StreamingLLM

Accuracy: 50. Result ID: 69aa961b6dbb7075b9547549.

[Reporting paper](<https://www.sota2.com/research/paper/6923e6a0d6b0dd334292f0f0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72d71e209131cea58d79.md>)

Recorded evaluation configuration: {"retention ratio ρ":"0.1","Backbone":"Qwen2.5-VL-7B-Instruct"}. This record does not assert independent reproduction.

### StreamingLLM

Accuracy: 50. Result ID: 69aa961b6dbb7075b954754a.

[Reporting paper](<https://www.sota2.com/research/paper/6923e6a0d6b0dd334292f0f0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72d71e209131cea58d79.md>)

Recorded evaluation configuration: {"retention ratio ρ":"0.05","Backbone":"Qwen2.5-VL-7B-Instruct"}. This record does not assert independent reproduction.

### Gemma3-27B

Accuracy: 49.7. Result ID: 69982ce95698665ec487eb99.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc2b423ea42265729d1f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa779f1e209131cea746cd.md>)

Recorded evaluation configuration: {"Model Size":"27B"}. This record does not assert independent reproduction.

### LLaVA-OneVision 7B, Dense

Accuracy: 49.22. Result ID: 69ba5b8447f34ce924a4083f.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31a70636e5bf4afc0941.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa756f1e209131cea67e58.md>)

[Implementation](<https://llava-vl.github.io/blog/llava-onevision>)

Recorded evaluation configuration: {"Backbone":"LLaVA-OneVision 7B","Sparsity":"Dense"}. This record does not assert independent reproduction.

### GLM-4.5V

Accuracy: 48.5. Result ID: 6a03491b47f34ce924e87990.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b479dbc836b87d824bfb.md>)

Recorded evaluation configuration: {"Zero-shot":"true","Parameters":"106B-A12B","Organization":"Z.ai"}. This record does not assert independent reproduction.

### Self-Refinement

Accuracy: 47.2. Result ID: 6996c8685698665ec41414c8.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f7f1e209131cea45fc4.md>)

Recorded evaluation configuration: {"Agent":"Ovis2"}. This record does not assert independent reproduction.

### Self-Refinement

Accuracy: 46.7. Result ID: 6996c8685698665ec41414c7.

[Reporting paper](<https://www.sota2.com/research/paper/6937ad116cfd488761a4e841.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f7f1e209131cea45fc4.md>)

Recorded evaluation configuration: {"Agent":"MiniCPM-o"}. This record does not assert independent reproduction.

### Phi-3-Vision-4B

Accuracy: 46.1. Result ID: 6a0ab5d547f34ce924f03b0a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77611e209131cea72d70.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74ae1e209131cea632e1.md>)

Recorded evaluation configuration: {"Model Scale":"4B","Model Access Type":"Open-source"}. This record does not assert independent reproduction.

### Gemma3-4B + AuditDM

Accuracy: 45.2. Result ID: 69982ce95698665ec487eb9b.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc2b423ea42265729d1f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6944dc2b423ea42265729d1f.md>)

Recorded evaluation configuration: {"Model Size":"4B","Auditing Component":"AuditDM"}. This record does not assert independent reproduction.

### LWM

Accuracy: 44.8. Result ID: 69954a633badfaffe6395cba.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77981e209131cea743dd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76d41e209131cea6f608.md>)

Recorded evaluation configuration: {"LLM Param":"7B","Type":"Unified"}. This record does not assert independent reproduction.

### Gemma3-12B

Accuracy: 44.8. Result ID: 69982ce95698665ec487eb98.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc2b423ea42265729d1f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa779f1e209131cea746cd.md>)

Recorded evaluation configuration: {"Model Size":"12B"}. This record does not assert independent reproduction.

### Full Cache

Accuracy: 43.34. Result ID: 69fc3e8947f34ce924de9a7d.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c5e5497b3b670c88334.md>)

Recorded evaluation configuration: {"Architecture":"LLaVA-v1.5","Parameter Scale":"7B"}. This record does not assert independent reproduction.

### PSFT

Accuracy: 43.33. Result ID: 69ddf9b647f34ce924c55b97.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ae21e209131cea85a73.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7ae21e209131cea85a73.md>)

Recorded evaluation configuration: {"Training Stage":"PSFT","RL Alignment (GRPO)":"false"}. This record does not assert independent reproduction.

### Llama 4 Maverick

Accuracy: 43. Result ID: 6a03491b47f34ce924e8798f.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b479dbc836b87d824bfb.md>)

Recorded evaluation configuration: {"Zero-shot":"true","Parameters":"400B-A17B","Organization":"MetaAI"}. This record does not assert independent reproduction.

### SMoE (k=2)

Accuracy: 42.67. Result ID: 6a33ac87fbc373592c30f39d.

[Reporting paper](<https://www.sota2.com/research/paper/6a337b91206542251d7c5083.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6cc11e209131cea35cdf.md>)

Recorded evaluation configuration: {"k":"2","Smoothing mechanism":"None","Backbone":"ViT-based","Model parameters":"5.6B","Fine-tuning dataset":"50% of the LLaVA-665K dataset","Number of experts":"4"}. This record does not assert independent reproduction.

### SPOT-E

Accuracy: 42.5. Result ID: 6a34fe4ffbc373592c325efe.

[Reporting paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a34cd17206542251d7c5f2e.md>)

Recorded evaluation configuration: {"Base Model":"LLaVA-OV-7B","Source Type":"Open-Source","Test-time Adaptation":"SPOT-E"}. This record does not assert independent reproduction.

### RetentiveKV

Accuracy: 42.45. Result ID: 69fc3e8a47f34ce924de9a83.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c5e5497b3b670c88334.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69fc1c5e5497b3b670c88334.md>)

Recorded evaluation configuration: {"Architecture":"LLaVA-v1.5","Parameter Scale":"7B"}. This record does not assert independent reproduction.

### SmoothSMoE annealed (k=2)

Accuracy: 42.44. Result ID: 6a33ac87fbc373592c30f39f.

[Reporting paper](<https://www.sota2.com/research/paper/6a337b91206542251d7c5083.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a337b91206542251d7c5083.md>)

Recorded evaluation configuration: {"k":"2","Smoothing mechanism":"Annealed SmoothSMoE","Backbone":"ViT-based","Model parameters":"5.6B","Fine-tuning dataset":"50% of the LLaVA-665K dataset","Number of experts":"4"}. This record does not assert independent reproduction.

### H2O

Accuracy: 42.39. Result ID: 69fc3e8947f34ce924de9a7e.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c5e5497b3b670c88334.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa718c1e209131cea51ead.md>)

[Implementation](<https://github.com/FMInference/H2O>)

Recorded evaluation configuration: {"Architecture":"LLaVA-v1.5","Parameter Scale":"7B"}. This record does not assert independent reproduction.

### Meda

Accuracy: 42.36. Result ID: 69fc3e8a47f34ce924de9a81.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c5e5497b3b670c88334.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77611e209131cea72d66.md>)

Recorded evaluation configuration: {"Architecture":"LLaVA-v1.5","Parameter Scale":"7B"}. This record does not assert independent reproduction.

### SnapKV

Accuracy: 42.33. Result ID: 69fc3e8947f34ce924de9a7f.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c5e5497b3b670c88334.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b21e209131cea5df89.md>)

[Implementation](<https://github.com/FasterDecoding/SnapKV>)

Recorded evaluation configuration: {"Architecture":"LLaVA-v1.5","Parameter Scale":"7B"}. This record does not assert independent reproduction.

### Full Cache

Accuracy: 42.33. Result ID: 69fc3e8a47f34ce924de9a84.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c5e5497b3b670c88334.md>)

Recorded evaluation configuration: {"Architecture":"Qwen3-VL","Parameter Scale":"8B"}. This record does not assert independent reproduction.

### SAINT

Accuracy: 42.28. Result ID: 69fc3e8a47f34ce924de9a80.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1c5e5497b3b670c88334.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa777a1e209131cea736d9.md>)

Recorded evaluation configuration: {"Architecture":"LLaVA-v1.5","Parameter Scale":"7B"}. This record does not assert independent reproduction.

### Qwen2-VL-2B

Accuracy: 42.2. Result ID: 6a0ab5d547f34ce924f03b0c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77611e209131cea72d70.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75151e209131cea65976.md>)

[Implementation](<https://github.com/QwenLM/Qwen2-VL>)

Recorded evaluation configuration: {"Model Scale":"2B","Model Access Type":"Open-source"}. This record does not assert independent reproduction.

Page 1 of 4. This is an index page, not the full collection.

- [Next index page](<https://www.sota2.com/research/sota/massive-multi-discipline-multimodal-understanding-on-mmmu.md?metric=Accuracy&page=2>): Follow this link to discover more pages.
