# MMU

Source: [MMU](<https://www.sota2.com/research/dataset/mmu>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Benchmarks

Page 1 of 7. This is an index page, not the full collection.

- [Next index page](<https://www.sota2.com/research/dataset/mmu.md?page=2>): Follow this link to discover more pages.

- [Multimodal Understanding on MMMU](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmmu.md>): Multimodal Understanding; MMMU.

- [Multi-discipline Multimodal Understanding on MMMU](<https://www.sota2.com/research/sota/multi-discipline-multimodal-understanding-on-mmmu.md>): Multi-discipline Multimodal Understanding; MMMU.

- [Massive Multi-discipline Multimodal Understanding on MMMU](<https://www.sota2.com/research/sota/massive-multi-discipline-multimodal-understanding-on-mmmu.md>): Massive Multi-discipline Multimodal Understanding; MMMU.

- [Multimodal Understanding on MMMU (Efficiency Metrics)](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmmu-efficiency-metrics.md>): Multimodal Understanding; MMMU.

- [Multimodal Reasoning on MMMU](<https://www.sota2.com/research/sota/multimodal-reasoning-on-mmmu.md>): Multimodal Reasoning; MMMU.

- [Multi-discipline Multimodal Understanding on MMMU (val)](<https://www.sota2.com/research/sota/multi-discipline-multimodal-understanding-on-mmmu-val.md>): Multi-discipline Multimodal Understanding; MMMU (val).

- [Multimodal Understanding on MMMU (val)](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmmu-val.md>): Multimodal Understanding; MMMU (val).

- [Multimodal Reasoning on MMMU Pro (Accuracy)](<https://www.sota2.com/research/sota/multimodal-reasoning-on-mmmu-pro-accuracy.md>): Multimodal Reasoning; MMMU Pro.

- [Multimodal Reasoning on MMMU (val)](<https://www.sota2.com/research/sota/multimodal-reasoning-on-mmmu-val.md>): Multimodal Reasoning; MMMU (val).

- [Video Question Answering on VideoMMMU](<https://www.sota2.com/research/sota/video-question-answering-on-videommmu.md>): Video Question Answering; VideoMMMU.

- [Video Reasoning on VideoMMMU](<https://www.sota2.com/research/sota/video-reasoning-on-videommmu.md>): Video Reasoning; VideoMMMU.

- [Multimodal Understanding on MMMU (test)](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmmu-test.md>): Multimodal Understanding; MMMU (test).

- [Multimodal Understanding on MMMU (Score)](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmmu-score.md>): Multimodal Understanding; MMMU.

- [Multimodal Understanding on MMMU (Accuracy, Average performance retention)](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmmu-accuracy-average-performance-retention.md>): Multimodal Understanding; MMMU.

- [Visual Question Answering on MMMU](<https://www.sota2.com/research/sota/visual-question-answering-on-mmmu.md>): Visual Question Answering; MMMU.

- [Multi-modal Question Answering on MMMU](<https://www.sota2.com/research/sota/multi-modal-question-answering-on-mmmu.md>): Multi-modal Question Answering; MMMU.

- [Multimodal Understanding on MMMU, SEED, OCRBench, VizWiz, ScienceQA, and TextVQA (test)](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmmu-seed-ocrbench-vizwiz-scienceqa-and-textvqa-test.md>): Multimodal Understanding; MMMU, SEED, OCRBench, VizWiz, ScienceQA, and TextVQA Average (test).

- [Multi-discipline Reasoning on MMMU](<https://www.sota2.com/research/sota/multi-discipline-reasoning-on-mmmu.md>): Multi-discipline Reasoning; MMMU.

- [Video Reasoning on Video-MMMU](<https://www.sota2.com/research/sota/video-reasoning-on-video-mmmu.md>): Video reasoning; Video-MMMU.

- [Regression on Communities and Crime 1990 US Census / 1990 US LEMAS / 1995 FBI UCR (test (20%))](<https://www.sota2.com/research/sota/regression-on-communities-and-crime-1990-us-census-1990-us-lemas-1995-fbi-ucr-test-20-percent.md>): Regression; Communities and Crime 1990 US Census / 1990 US LEMAS / 1995 FBI UCR (test (20%)).

- [Multimodal Reasoning on MMMU (Accuracy)](<https://www.sota2.com/research/sota/multimodal-reasoning-on-mmmu-accuracy.md>): Multimodal Reasoning; MMMU.

- [Accuracy on MMMU](<https://www.sota2.com/research/sota/accuracy-on-mmmu.md>): Multimodal Understanding; MMMU.

- [Vision Understanding on MMMU](<https://www.sota2.com/research/sota/vision-understanding-on-mmmu.md>): Vision Understanding; MMMU.

- [Multimodal Understanding on MMMU (MMMU Score)](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmmu-mmmu-score.md>): Multimodal Understanding; MMMU.

- [Video Understanding on VideoMMMU](<https://www.sota2.com/research/sota/video-understanding-on-videommmu.md>): Video Understanding; VideoMMMU.

- [Multi-discipline Multimodal Understanding on MMMU Pro](<https://www.sota2.com/research/sota/multi-discipline-multimodal-understanding-on-mmmu-pro.md>): Multi-discipline Multimodal Understanding; MMMU Pro.

- [General Reasoning on MMMU](<https://www.sota2.com/research/sota/general-reasoning-on-mmmu.md>): General Reasoning; MMMU.

- [Node Classification on community (test)](<https://www.sota2.com/research/sota/node-classification-on-community-test.md>): Node Classification; community (test).

- [Multi-modal Understanding on LLaVA Multi-modal Evaluation Suite v1.6 (test)](<https://www.sota2.com/research/sota/multi-modal-understanding-on-llava-multi-modal-evaluation-suite-v1-6-test.md>): Multi-modal Understanding; LLaVA Multi-modal Evaluation Suite (GQA, MMB, MME, POPE, SQA, VQAv2, TextVQA, MMMU, SEED-I) v1.6 (test).

- [Video Multimodal Understanding on VideoMMMU](<https://www.sota2.com/research/sota/video-multimodal-understanding-on-videommmu.md>): Video Multimodal Understanding; VideoMMMU.

- [Multi-agent discussion attack on MMMU](<https://www.sota2.com/research/sota/multi-agent-discussion-attack-on-mmmu.md>): Multi-agent discussion attack; MMMU.

- [Visual Question Answering on LLaVA Evaluation Suite (GQA, MMB, MMMU, MME, VQAv2, VQAtext, POPE, SQA, SEED, MMVet)](<https://www.sota2.com/research/sota/visual-question-answering-on-llava-evaluation-suite-gqa-mmb-mmmu-mme-vqav2-vqatext-pope-sqa-seed-mmvet.md>): Visual Question Answering; LLaVA Evaluation Suite GQA, MMB, MMMU, MME, VQAv2, VQAtext, POPE, SQA, SEED, MMVet 1.5 7B (various).

- [Multiple Choice Question Answering on PathMMU (val)](<https://www.sota2.com/research/sota/multiple-choice-question-answering-on-pathmmu-val.md>): Multiple Choice Question Answering; PathMMU (val).

- [Personalization on Community Alignment (CA)](<https://www.sota2.com/research/sota/personalization-on-community-alignment-ca.md>): Personalization; Community Alignment (CA).

- [Multimodal Reasoning on MMMU (test)](<https://www.sota2.com/research/sota/multimodal-reasoning-on-mmmu-test.md>): Multimodal Reasoning; MMMU (test).

- [Multimodal Understanding and Reasoning on MMMU, SEED, OCRBench, VizWiz, ScienceQA, and TextVQA (test/val)](<https://www.sota2.com/research/sota/multimodal-understanding-and-reasoning-on-mmmu-seed-ocrbench-vizwiz-scienceqa-and-textvqa-test-val.md>): Multimodal Understanding and Reasoning; MMMU, SEED, OCRBench, VizWiz, ScienceQA, TextVQA (test/val).

- [Visual Question Answering on PathMMU Tiny 1.0 (test)](<https://www.sota2.com/research/sota/visual-question-answering-on-pathmmu-tiny-1-0-test.md>): Visual Question Answering; PathMMU Tiny 1.0 (test).

- [Multimodal Reasoning on MMMU (Acc.%, Acpt.%, Speedup)](<https://www.sota2.com/research/sota/multimodal-reasoning-on-mmmu-acc-percent-acpt-percent-speedup.md>): Multimodal Reasoning; MMMU.

- [Medical Visual Question Answering on MMMU Health & Medicine (test)](<https://www.sota2.com/research/sota/medical-visual-question-answering-on-mmmu-health-and-medicine-test.md>): Medical Visual Question Answering; MMMU Health & Medicine (test).

- [Multimodal Reasoning on MMMU-Pro offline (test)](<https://www.sota2.com/research/sota/multimodal-reasoning-on-mmmu-pro-offline-test.md>): Multimodal Reasoning; MMMU-Pro offline (test).

- [Multimodal Reasoning on MMMU (offline) (Accuracy)](<https://www.sota2.com/research/sota/multimodal-reasoning-on-mmmu-offline-accuracy.md>): Multimodal Reasoning; MMMU (offline).

- [Multimodal Understanding on LVLM Evaluation Suite (AI2D, DocVQA, InfoVQA, MMBench, MME, MMMU, SciQA, TextVQA, MMStar, POPE)](<https://www.sota2.com/research/sota/multimodal-understanding-on-lvlm-evaluation-suite-ai2d-docvqa-infovqa-mmbench-mme-mmmu-sciqa-textvqa-mmstar-pope.md>): Multimodal Understanding; LVLM Evaluation Suite (AI2D, DocVQA, InfoVQA, MMBench, MME, MMMU, SciQA, TextVQA, MMStar, POPE).

- [Multimodal Understanding on MMMU (Accuracy)](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmmu-accuracy.md>): Multimodal Understanding; MMMU.

- [Multi-discipline reasoning on MMMU (val) (Accuracy)](<https://www.sota2.com/research/sota/multi-discipline-reasoning-on-mmmu-val-accuracy.md>): Multi-discipline reasoning; MMMU (val).

- [Visual Question Answering on MMMU (val)](<https://www.sota2.com/research/sota/visual-question-answering-on-mmmu-val.md>): Visual Question Answering; MMMU (val).

- [Multimodal Evaluation on MMMU](<https://www.sota2.com/research/sota/multimodal-evaluation-on-mmmu.md>): Multimodal Evaluation; MMMU.

- [Multi-modal Reasoning on MMMU-Pro](<https://www.sota2.com/research/sota/multi-modal-reasoning-on-mmmu-pro.md>): Multi-modal Reasoning; MMMU-Pro.

- [General Visual Understanding on MMMU](<https://www.sota2.com/research/sota/general-visual-understanding-on-mmmu.md>): General Visual Understanding; MMMU.

- [Multi-modal Understanding on MMMU (Accuracy)](<https://www.sota2.com/research/sota/multi-modal-understanding-on-mmmu-accuracy.md>): Multi-modal Understanding; MMMU.

- [Multimodal Understanding on MMMU (Accuracy, Rel. (%))](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmmu-accuracy-rel-percent.md>): Multimodal Understanding; MMMU.

- [Multi-modal Understanding on MMMU Pro (Overall)](<https://www.sota2.com/research/sota/multi-modal-understanding-on-mmmu-pro-overall.md>): Multi-modal Understanding; MMMU Pro (Overall).

- [Video Understanding on MVBench, EgoSchema, VMME, LVB, VMMMU](<https://www.sota2.com/research/sota/video-understanding-on-mvbench-egoschema-vmme-lvb-vmmmu.md>): Video Understanding; Video Understanding Suite MVBench, EgoSchema, VMME, LVB, VMMMU.

- [Multimodal Reasoning on MMMU-Pro (Acc, # Tokens)](<https://www.sota2.com/research/sota/multimodal-reasoning-on-mmmu-pro-acc-sharp-tokens.md>): Multimodal Reasoning; MMMU-Pro.

- [Multi-discipline Multimodal Reasoning on MMMU](<https://www.sota2.com/research/sota/multi-discipline-multimodal-reasoning-on-mmmu.md>): Multi-discipline Multimodal Reasoning; MMMU.

- [Multimodal Knowledge and Math on MMMU (val)](<https://www.sota2.com/research/sota/multimodal-knowledge-and-math-on-mmmu-val.md>): Multimodal Knowledge and Math; MMMU (val).

- [Prevalence Estimation on ACS (American Community Survey) Out-of-Distribution](<https://www.sota2.com/research/sota/prevalence-estimation-on-acs-american-community-survey-out-of-distribution.md>): Prevalence Estimation; ACS (American Community Survey) Out-of-Distribution.

- [Prevalence Estimation on ACS (American Community Survey) In-Distribution](<https://www.sota2.com/research/sota/prevalence-estimation-on-acs-american-community-survey-in-distribution.md>): Prevalence Estimation; ACS (American Community Survey) In-Distribution.

- [Visual Reasoning on MMMU-Pro (ECE)](<https://www.sota2.com/research/sota/visual-reasoning-on-mmmu-pro-ece.md>): Visual Reasoning; MMMU-Pro.

- [Over-refusal evaluation on MMMU (test)](<https://www.sota2.com/research/sota/over-refusal-evaluation-on-mmmu-test.md>): Over-refusal evaluation; MMMU in-scope (test).

- [General Multimodal Understanding on MMMU Pro](<https://www.sota2.com/research/sota/general-multimodal-understanding-on-mmmu-pro.md>): General Multimodal Understanding; MMMU Pro.

- [General Visual Understanding on MMMU Pro](<https://www.sota2.com/research/sota/general-visual-understanding-on-mmmu-pro.md>): General Visual Understanding; MMMU Pro.

- [Vision-dependent Multimodal Reasoning on Vision-dependent Multimodal Reasoning Suite (LogicVista, Counting, MMMU-Pro, MathVerseV)](<https://www.sota2.com/research/sota/vision-dependent-multimodal-reasoning-on-vision-dependent-multimodal-reasoning-suite-logicvista-counting-mmmu-pro-mathversev.md>): Vision-dependent Multimodal Reasoning; Vision-dependent Multimodal Reasoning Suite (LogicVista, Counting, MMMU-Pro, MathVerseV).

- [Medical Visual Question Answering on MMMU Med (Accuracy)](<https://www.sota2.com/research/sota/medical-visual-question-answering-on-mmmu-med-accuracy.md>): Medical Visual Question Answering; MMMU Med.

- [Visual Reasoning on MMMU-Pro](<https://www.sota2.com/research/sota/visual-reasoning-on-mmmu-pro.md>): Visual Reasoning; MMMU-Pro.

- [Video Understanding and Reasoning on Video-MMMU (test)](<https://www.sota2.com/research/sota/video-understanding-and-reasoning-on-video-mmmu-test.md>): Video Understanding and Reasoning; Video-MMMU (test).

- [Video Understanding on Video-MMMU](<https://www.sota2.com/research/sota/video-understanding-on-video-mmmu.md>): Video Understanding; Video-MMMU.

- [Cognition and Reasoning on MMMU (val)](<https://www.sota2.com/research/sota/cognition-and-reasoning-on-mmmu-val.md>): Cognition and Reasoning; MMMU (val).

- [Video Reasoning on VideoMMMU comprehension](<https://www.sota2.com/research/sota/video-reasoning-on-videommmu-comprehension.md>): Video Reasoning; VideoMMMU comprehension.

- [Video Reasoning on VideoMMMU adaptation](<https://www.sota2.com/research/sota/video-reasoning-on-videommmu-adaptation.md>): Video Reasoning; VideoMMMU adaptation.

- [Image Reasoning on MMMU Pro Vision](<https://www.sota2.com/research/sota/image-reasoning-on-mmmu-pro-vision.md>): Image Reasoning; MMMU Pro Vision.

- [Adversarial Attack Success Rate on Multi-agent Communication Scenarios (test)](<https://www.sota2.com/research/sota/adversarial-attack-success-rate-on-multi-agent-communication-scenarios-test.md>): Adversarial Attack Success Rate; Multi-agent Communication Scenarios (test).

- [Multimodal Reasoning on MMMU (Efficiency and Performance Metrics)](<https://www.sota2.com/research/sota/multimodal-reasoning-on-mmmu-efficiency-and-performance-metrics.md>): Multimodal Reasoning; MMMU.

- [Medical Visual Question Answering on MMMU Med](<https://www.sota2.com/research/sota/medical-visual-question-answering-on-mmmu-med.md>): Medical Visual Question Answering; MMMU Med.

- [Multimodal Question Answering and Understanding on Standard Multimodal Evaluation Suite (GQA, MMB, MME, VQA-T, SQA-I, VQA-v2, POPE, MMMU, MM-Vet)](<https://www.sota2.com/research/sota/multimodal-question-answering-and-understanding-on-standard-multimodal-evaluation-suite-gqa-mmb-mme-vqa-t-sqa-i-vqa-v2-pope-mmmu-mm-vet.md>): Multimodal Question Answering and Understanding; Standard Multimodal Evaluation Suite GQA, MMB, MME, VQA-T, SQA-I, VQA-v2, POPE, MMMU, MM-Vet.

- [Multimodal Image Understanding on MMMU](<https://www.sota2.com/research/sota/multimodal-image-understanding-on-mmmu.md>): Multimodal Image Understanding; MMMU.

- [Visual Question Answering on MMMU-Pro](<https://www.sota2.com/research/sota/visual-question-answering-on-mmmu-pro.md>): Visual Question Answering; MMMU-Pro.

- [STEM & Puzzle on MMMU (val)](<https://www.sota2.com/research/sota/stem-and-puzzle-on-mmmu-val.md>): STEM & Puzzle; MMMU (val).

- [Multimodal Understanding on MMMU zero-shot](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmmu-zero-shot.md>): Multimodal Understanding; MMMU zero-shot.

- [Graph Generation on Community-small](<https://www.sota2.com/research/sota/graph-generation-on-community-small.md>): Graph Generation; Community-small.

- [Multimodal Video Understanding on VideoMMMU](<https://www.sota2.com/research/sota/multimodal-video-understanding-on-videommmu.md>): Multimodal Video Understanding; VideoMMMU.
