# Multimodal Understanding on MMB, MME, SQA, VQA^T, MMB^C, MMVet, MMstar, AI2D Suite Source: [Multimodal Understanding on MMB, MME, SQA, VQA^T, MMB^C, MMVet, MMstar, AI2D Suite]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:49:57.126Z Aggregate evaluation covering performance across the MMB, MME, SQA, VQA^T, MMB^C, MMVet, MMstar, and AI2D multimodal understanding benchmarks. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multimodal Understanding | | Dataset (including any stated split) | Multimodal Evaluation Suite MMB, MME, SQA, VQA^T, MMB^C, MMVet, MMstar, AI2D | | Metric | MMB Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 17 of 17 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [MMB Score](): Higher is better; unit: PERCENT. - [MME Score](): Higher is better; unit: SCALAR. - [SQA Score](): Higher is better; unit: PERCENT. - [VQA^T Score](): Higher is better; unit: PERCENT. - [MMB^C Score](): Higher is better; unit: PERCENT. - [MMVet Score](): Higher is better; unit: PERCENT. - [MMstar Score](): Higher is better; unit: PERCENT. - [AI2D Score](): Higher is better; unit: PERCENT. - [Accuracy](): Higher is better; unit: PERCENT. - [Rel Score (%)](): Higher is better; unit: PERCENT. ## Recorded results | Method | MMB Score | Result date | Report paper | | --- | --- | --- | --- | | LLaVA-1.5-7B | 64.6 | Not recorded | 6997ea7209c33900fcd76502 | | EntropyPrune | 64.4 | Not recorded | 6997ea7209c33900fcd76502 | | DART | 63.7 | Not recorded | 6997ea7209c33900fcd76502 | | EntropyPrune | 63.5 | Not recorded | 6997ea7209c33900fcd76502 | | SparseVLM | 63.4 | Not recorded | 6997ea7209c33900fcd76502 | | PDrop | 63.3 | Not recorded | 6997ea7209c33900fcd76502 | | FastV | 63.2 | Not recorded | 6997ea7209c33900fcd76502 | | CDPruner | 63.2 | Not recorded | 6997ea7209c33900fcd76502 | | SparseVLM | 63.2 | Not recorded | 6997ea7209c33900fcd76502 | | CDPruner | 63 | Not recorded | 6997ea7209c33900fcd76502 | | FastV | 62.5 | Not recorded | 6997ea7209c33900fcd76502 | | DivPrune | 62.3 | Not recorded | 6997ea7209c33900fcd76502 | | PDrop | 62.2 | Not recorded | 6997ea7209c33900fcd76502 | | DART | 62.1 | Not recorded | 6997ea7209c33900fcd76502 | | DivPrune | 61.3 | Not recorded | 6997ea7209c33900fcd76502 | | Prumerge | 56.3 | Not recorded | 6997ea7209c33900fcd76502 | | Prumerge | 54.6 | Not recorded | 6997ea7209c33900fcd76502 | ### LLaVA-1.5-7B MMB Score: 64.6. Result ID: 699b08665698665ec49afb04. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Retained Tokens":"576","Reduction Rate":"0%","FLOPs":"8.80T"}. This record does not assert independent reproduction. ### EntropyPrune MMB Score: 64.4. Result ID: 699b08665698665ec49afb0c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retained Tokens":"192","Reduction Rate":"66.7%","FLOPs":"3.72T"}. This record does not assert independent reproduction. ### DART MMB Score: 63.7. Result ID: 699b08665698665ec49afb08. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retained Tokens":"192","Reduction Rate":"66.7%","FLOPs":"3.69T"}. This record does not assert independent reproduction. ### EntropyPrune MMB Score: 63.5. Result ID: 699b08665698665ec49afb14. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retained Tokens":"128","Reduction Rate":"77.8%","FLOPs":"2.88T"}. This record does not assert independent reproduction. ### SparseVLM MMB Score: 63.4. Result ID: 699b08665698665ec49afb07. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retained Tokens":"192","Reduction Rate":"66.7%","FLOPs":"3.76T"}. This record does not assert independent reproduction. ### PDrop MMB Score: 63.3. Result ID: 699b08665698665ec49afb06. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retained Tokens":"192","Reduction Rate":"66.7%","FLOPs":"3.79T"}. This record does not assert independent reproduction. ### FastV MMB Score: 63.2. Result ID: 699b08665698665ec49afb05. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retained Tokens":"192","Reduction Rate":"66.7%","FLOPs":"3.99T"}. This record does not assert independent reproduction. ### CDPruner MMB Score: 63.2. Result ID: 699b08665698665ec49afb09. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retained Tokens":"192","Reduction Rate":"66.7%","FLOPs":"3.76T"}. This record does not assert independent reproduction. ### SparseVLM MMB Score: 63.2. Result ID: 699b08665698665ec49afb0f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retained Tokens":"128","Reduction Rate":"77.8%","FLOPs":"2.92T"}. This record does not assert independent reproduction. ### CDPruner MMB Score: 63. Result ID: 699b08665698665ec49afb11. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retained Tokens":"128","Reduction Rate":"77.8%","FLOPs":"2.92T"}. This record does not assert independent reproduction. ### FastV MMB Score: 62.5. Result ID: 699b08665698665ec49afb0d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retained Tokens":"128","Reduction Rate":"77.8%","FLOPs":"3.14T"}. This record does not assert independent reproduction. ### DivPrune MMB Score: 62.3. Result ID: 699b08665698665ec49afb0a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retained Tokens":"192","Reduction Rate":"66.7%","FLOPs":"3.73T"}. This record does not assert independent reproduction. ### PDrop MMB Score: 62.2. Result ID: 699b08665698665ec49afb0e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retained Tokens":"128","Reduction Rate":"77.8%","FLOPs":"2.98T"}. This record does not assert independent reproduction. ### DART MMB Score: 62.1. Result ID: 699b08665698665ec49afb10. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retained Tokens":"128","Reduction Rate":"77.8%","FLOPs":"2.80T"}. This record does not assert independent reproduction. ### DivPrune MMB Score: 61.3. Result ID: 699b08665698665ec49afb12. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retained Tokens":"128","Reduction Rate":"77.8%","FLOPs":"2.88T"}. This record does not assert independent reproduction. ### Prumerge MMB Score: 56.3. Result ID: 699b08665698665ec49afb0b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retained Tokens":"192","Reduction Rate":"66.7%","FLOPs":"3.71T"}. This record does not assert independent reproduction. ### Prumerge MMB Score: 54.6. Result ID: 699b08665698665ec49afb13. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retained Tokens":"128","Reduction Rate":"77.8%","FLOPs":"2.87T"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.