# Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters More

Source: [Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters More](<https://www.sota2.com/research/paper/68fa79461e209131cea7cfd7>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T04:55:51.332Z

Authors: Zichen Wen, Yifeng Gao, Shaobo Wang, Junyuan Zhang, Qintong Zhang, Weijia Li, Conghui He, Linfeng Zhang

Published: 2025-02-17T06:56:28.000Z

## Abstract

Vision tokens in multimodal large language models often dominate huge computational overhead due to their excessive length compared to linguistic modality. Abundant recent methods aim to solve this problem with token pruning, which first defines an importance criterion for tokens and then prunes the unimportant vision tokens during inference. However, in this paper, we show that the importance is not an ideal indicator to decide whether a token should be pruned. Surprisingly, it usually results in inferior performance than random token pruning and leading to incompatibility to efficient attention computation operators.Instead, we propose DART (Duplication-Aware Reduction of Tokens), which prunes tokens based on its duplication with other tokens, leading to significant and training-free acceleration. Concretely, DART selects a small subset of pivot tokens and then retains the tokens with low duplication to the pivots, ensuring minimal information loss during token pruning. Experiments demonstrate that DART can prune 88.9% vision tokens while maintaining comparable performance, leading to a 1.99$\\times$ and 2.99$\\times$ speed-up in total time and prefilling stage, respectively, with good compatibility to efficient attention operators. Our codes are available at https://github.com/ZichenWen1/DART.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Visual Question Answering on ScienceQA SQA-IMG](<https://www.sota2.com/research/sota/visual-question-answering-on-scienceqa-sqa-img.md>): Visual Question Answering; ScienceQA SQA-IMG; SQAIMG Accuracy: 74.4.

- [Science Question Answering on ScienceQA Image](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-image.md>): Science Question Answering; ScienceQA image; Relative Performance: 100.7.

- [Multimodal Evaluation on MME (MME, Rel. Acc)](<https://www.sota2.com/research/sota/multimodal-evaluation-on-mme-mme-rel-acc.md>): Multimodal Evaluation; MME; MME Score: 1534.8.

- [Visual Question Answering on TextVQA (Accuracy, Rel. Acc)](<https://www.sota2.com/research/sota/visual-question-answering-on-textvqa-accuracy-rel-acc.md>): Visual Question Answering; TextVQA; Accuracy: 64.7.

- [Visual Language Model Evaluation on MMVet](<https://www.sota2.com/research/sota/visual-language-model-evaluation-on-mmvet.md>): Visual Language Model Evaluation; MMVet; MMVet Score: 40.6.

- [Multi-modal Benchmarking on MMBench Chinese](<https://www.sota2.com/research/sota/multi-modal-benchmarking-on-mmbench-chinese.md>): Multi-modal Benchmarking; MMBench Chinese; Accuracy: 83.2.

- [Audio Understanding on AudioMarathon](<https://www.sota2.com/research/sota/audio-understanding-on-audiomarathon.md>): Audio Understanding; AudioMarathon; SER: 16.8.

- [Multi-modal Evaluation on MME (Cognition and Perception Scores)](<https://www.sota2.com/research/sota/multi-modal-evaluation-on-mme-cognition-and-perception-scores.md>): Multi-modal Evaluation; MME; MME Perception Score: 1724.

- [Multimodal Question Answering on MMBench en (test)](<https://www.sota2.com/research/sota/multimodal-question-answering-on-mmbench-en-test.md>): Multimodal Question Answering; MMBench EN (test); Accuracy: 85.

- [Multimodal Understanding on AI2D, POPE, TextVQA, OKVQA, VizWiz, COCO Caption, NoCaps, and RealWorldQA](<https://www.sota2.com/research/sota/multimodal-understanding-on-ai2d-pope-textvqa-okvqa-vizwiz-coco-caption-nocaps-and-realworldqa.md>): Multimodal Understanding; AI2D, POPE, TextVQA, OKVQA, VizWiz, COCO Caption, NoCaps, and RealWorldQA Average; Average Score: 99.16.

- [Visual Question Answering on GQA (GQA, Rel. Acc)](<https://www.sota2.com/research/sota/visual-question-answering-on-gqa-gqa-rel-acc.md>): Visual Question Answering; GQA; GQA Score: 61.

- [Multimodal Evaluation on MME (Score, Rel. Acc)](<https://www.sota2.com/research/sota/multimodal-evaluation-on-mme-score-rel-acc.md>): Multimodal Evaluation; MME; MME Score: 1558.1.

- [Multimodal Understanding and Reasoning on Multimodal Evaluation Suite (VQAv2, GQA, SQAImg, VQAText, POPE, MME, MMBEN, MMBCN, MMVet)](<https://www.sota2.com/research/sota/multimodal-understanding-and-reasoning-on-multimodal-evaluation-suite-vqav2-gqa-sqaimg-vqatext-pope-mme-mmben-mmbcn-mmvet.md>): Multimodal Understanding and Reasoning; Multimodal Evaluation Suite (VQAv2, GQA, SQAImg, VQAText, POPE, MME, MMBEN, MMBCN, MMVet); VQAv2 Accuracy: 80.6.

- [Performance Compression Stability on Aggregate Benchmarks](<https://www.sota2.com/research/sota/performance-compression-stability-on-aggregate-benchmarks.md>): Performance Compression Stability; Aggregate Benchmarks; Relative Performance: 99.3.

- [Visual Question Answering on VQA v2 (Accuracy and Relative Accuracy)](<https://www.sota2.com/research/sota/visual-question-answering-on-vqa-v2-accuracy-and-relative-accuracy.md>): Visual Question Answering; VQA v2; Accuracy: 81.8.

- [Visual Question Answering on GQA (Accuracy and Rel. Acc)](<https://www.sota2.com/research/sota/visual-question-answering-on-gqa-accuracy-and-rel-acc.md>): Visual Question Answering; GQA; Accuracy: 64.4.

- [Multimodal Question Answering on MMBench CN (test)](<https://www.sota2.com/research/sota/multimodal-question-answering-on-mmbench-cn-test.md>): Multimodal Question Answering; MMBench CN (test); Accuracy: 85.4.

- [OCR Evaluation on OCRBench (test)](<https://www.sota2.com/research/sota/ocr-evaluation-on-ocrbench-test.md>): OCR Evaluation; OCRBench (test); Score: 50.

- [Visual Language Model Evaluation on MM-Bench EN](<https://www.sota2.com/research/sota/visual-language-model-evaluation-on-mm-bench-en.md>): Visual Language Model Evaluation; MM-Bench EN; MM-Bench (EN) Score: 65.4.

- [Visual Question Answering on VizWiz (test)](<https://www.sota2.com/research/sota/visual-question-answering-on-vizwiz-test.md>): Visual Question Answering; VizWiz (test); Accuracy: 88.1.

- [Image Understanding on MMBench CN](<https://www.sota2.com/research/sota/image-understanding-on-mmbench-cn.md>): Image Understanding; MMBench CN; Accuracy: 58.2.

- [Visual Question Answering on VQAT](<https://www.sota2.com/research/sota/visual-question-answering-on-vqat.md>): Visual Question Answering; VQAT; Accuracy: 0.9798.

- [Multi-modal Understanding on MMBench (Language Scores)](<https://www.sota2.com/research/sota/multi-modal-understanding-on-mmbench-language-scores.md>): Multi-modal Understanding; MMBench; MMB Score (EN): 86.

- [Image Understanding Evaluation on MME](<https://www.sota2.com/research/sota/image-understanding-evaluation-on-mme.md>): Image Understanding Evaluation; MME; MME Score: 1844.

- [Multimodal Evaluation on MME (Absolute Score, Retention Rate, Avg Acc.)](<https://www.sota2.com/research/sota/multimodal-evaluation-on-mme-absolute-score-retention-rate-avg-acc.md>): Multimodal Evaluation; MME; Absolute Score: 1758.02.

- [Multimodal Understanding on MMB, MME, SQA, VQA^T, MMB^C, MMVet, MMstar, AI2D Suite](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmb-mme-sqa-vqa-t-mmb-c-mmvet-mmstar-ai2d-suite.md>): Multimodal Understanding; Multimodal Evaluation Suite MMB, MME, SQA, VQA^T, MMB^C, MMVet, MMstar, AI2D; MMB Score: 63.7.

- [Multimodal Reasoning on RealWorldQA (test)](<https://www.sota2.com/research/sota/multimodal-reasoning-on-realworldqa-test.md>): Multimodal Reasoning; RealWorldQA (test); Accuracy: 54.25.

- [OCR-based Visual Question Answering on TextVQA](<https://www.sota2.com/research/sota/ocr-based-visual-question-answering-on-textvqa.md>): OCR-based Visual Question Answering; TextVQA; VQAText: 59.

- [Image Understanding on GQA, MME, POPE, VQA^T, MMB, SQA](<https://www.sota2.com/research/sota/image-understanding-on-gqa-mme-pope-vqa-t-mmb-sqa.md>): Image Understanding; Image Understanding benchmarks GQA, MME, POPE, VQA^T, MMB, SQA; GQA Score: 60.2.

- [Science Question Answering on ScienceQA IMG (EM)](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-img-em.md>): Science Question Answering; ScienceQA IMG; EM: 68.96.

- [Multimodal Understanding on LLaVA-NEXT-7B Evaluation Suite (test)](<https://www.sota2.com/research/sota/multimodal-understanding-on-llava-next-7b-evaluation-suite-test.md>): Multimodal Understanding; VLM Evaluation Suite (GQA, MMB, MMBCN, MME, POPE, SQA, VQAv2, VQAText) LLaVA-NEXT-7B (test); GQA Accuracy: 61.7.

- [Visual Language Model Evaluation on MM-Bench CN](<https://www.sota2.com/research/sota/visual-language-model-evaluation-on-mm-bench-cn.md>): Visual Language Model Evaluation; MM-Bench CN; MMB (CN) Score: 56.7.

- [Visual Question Answering on VQAv2 (NAR Metric)](<https://www.sota2.com/research/sota/visual-question-answering-on-vqav2-nar-metric.md>): Visual Question Answering; VQAv2; VQAv2 Score: 79.1.

- [Real-world Visual Reasoning on RealWorld](<https://www.sota2.com/research/sota/real-world-visual-reasoning-on-realworld.md>): Real World visual reasoning; RealWorld; Accuracy: 55.56.

- [Multi-modal Understanding on MMB EN](<https://www.sota2.com/research/sota/multi-modal-understanding-on-mmb-en.md>): Multi-modal Understanding; MMB EN; Performance Score: 78.2.

- [Visual Question Answering on VizWiz](<https://www.sota2.com/research/sota/visual-question-answering-on-vizwiz.md>): Visual Question Answering; VizWiz; Accuracy: 98.57.

- [Scientific Question Answering on ScienceQA image](<https://www.sota2.com/research/sota/scientific-question-answering-on-scienceqa-image.md>): Scientific Question Answering; ScienceQA image; Accuracy: 93.8.

- [Multimodal Understanding on LLaVA Evaluation Suite (GQA, MMB, MMB-CN, MME, POPE, SQA, VQAV2, VQAText, VizWiz)](<https://www.sota2.com/research/sota/multimodal-understanding-on-llava-evaluation-suite-gqa-mmb-mmb-cn-mme-pope-sqa-vqav2-vqatext-vizwiz.md>): Multimodal Understanding; LLaVA Evaluation Suite GQA, MMB, MMB-CN, MME, POPE, SQA, VQAV2, VQAText, VizWiz; GQA: 61.7.

- [Visual Question Answering on VQA v2 (Score)](<https://www.sota2.com/research/sota/visual-question-answering-on-vqa-v2-score.md>): Visual Question Answering; VQA v2; VQA-2 Score: 76.7.

- [Visual Question Answering on VQA v2 (Absolute Score, Retention Rate)](<https://www.sota2.com/research/sota/visual-question-answering-on-vqa-v2-absolute-score-retention-rate.md>): Visual Question Answering; VQA v2; Absolute Score: 76.8.

- [Multi-modal Understanding on MME (Score, Rel. Perf.)](<https://www.sota2.com/research/sota/multi-modal-understanding-on-mme-score-rel-perf.md>): Multi-modal Understanding; MME; MME Score: 1793.

- [Science Question Answering on SQA](<https://www.sota2.com/research/sota/science-question-answering-on-sqa.md>): Science Question Answering; SQA; Exact Match: 93.11.

- [Multimodal Understanding on MLLM Evaluation Suite (GQA, MME, POPE, SQA, VQAtext, VizWiz, MMBen, AI2D) (test)](<https://www.sota2.com/research/sota/multimodal-understanding-on-mllm-evaluation-suite-gqa-mme-pope-sqa-vqatext-vizwiz-mmben-ai2d-test.md>): Multimodal Understanding; MLLM Evaluation Suite GQA MME POPE SQA VQAtext VizWiz MMBen AI2D v1.5 v1.6 Qwen2.5 (test); GQA Score: 62.4.

- [Driving VQA on DriveLM (test)](<https://www.sota2.com/research/sota/driving-vqa-on-drivelm-test.md>): Driving VQA; DriveLM (test); Accuracy: 78.

- [Knowledge-based Question Answering on OKVQA](<https://www.sota2.com/research/sota/knowledge-based-question-answering-on-okvqa.md>): Knowledge-based Question Answering; OKVQA; Score: 63.83.

- [Vision-Language Perception and Cognition on MME](<https://www.sota2.com/research/sota/vision-language-perception-and-cognition-on-mme.md>): Vision-Language Perception and Cognition; MME; Total Score: 1710.

- [Visual Question Answering on GQA (GQA Score, Relative Metric)](<https://www.sota2.com/research/sota/visual-question-answering-on-gqa-gqa-score-relative-metric.md>): Visual Question Answering; GQA; GQA Score: 59.4.

- [Image Captioning on NoCaps (Generic Score)](<https://www.sota2.com/research/sota/image-captioning-on-nocaps-generic-score.md>): Image Captioning; NoCaps; Score: 78.89.

- [Multimodal Question Answering on LLaVA-NeXT-7B Evaluation Suite](<https://www.sota2.com/research/sota/multimodal-question-answering-on-llava-next-7b-evaluation-suite.md>): Multimodal Question Answering; LLaVA-Next-7B Evaluation Suite; FLOPs (TFLOPs): 8.6.

- [Robot Manipulation on RoboTwin (OOD)](<https://www.sota2.com/research/sota/robot-manipulation-on-robotwin-ood.md>): Robot Manipulation; RoboTwin2 (OOD); Adjust Bottle Success Rate: 40.

- [Multi-modal Evaluation on MME (Score Metric)](<https://www.sota2.com/research/sota/multi-modal-evaluation-on-mme-score-metric.md>): Multi-modal Evaluation; MME; MME Score: 2280.4.

- [Optical Character Recognition Evaluation on OCRBench](<https://www.sota2.com/research/sota/optical-character-recognition-evaluation-on-ocrbench.md>): Optical Character Recognition Evaluation; OCRBench; Score: 805.

- [Visual Question Answering on ScienceQA image](<https://www.sota2.com/research/sota/visual-question-answering-on-scienceqa-image.md>): Visual Question Answering; ScienceQA image; Accuracy: 74.2.

- [Audio Understanding on MMAU Pro](<https://www.sota2.com/research/sota/audio-understanding-on-mmau-pro.md>): Audio Understanding; MMAU-Pro; Average Score: 98.7.

- [Multimodal Evaluation on MMB](<https://www.sota2.com/research/sota/multimodal-evaluation-on-mmb.md>): Multimodal Evaluation; MMB; Score: 79.81.

- [Vision-Language Understanding on Vision-Language Benchmark Suite Aggregate](<https://www.sota2.com/research/sota/vision-language-understanding-on-vision-language-benchmark-suite-aggregate.md>): Vision-Language Understanding; Vision-Language Benchmark Suite Aggregate; Aggregate Performance (%): 98.3.

- [Localization on RefCOCO+ (val)](<https://www.sota2.com/research/sota/localization-on-refcoco-plus-val.md>): Localization; RefCOCO+ (val); Accuracy: 80.91.

- [Visual Question Answering on GQA v1.2 (test)](<https://www.sota2.com/research/sota/visual-question-answering-on-gqa-v1-2-test.md>): Visual Question Answering; GQA v1.2 (test); GQA Score: 60.

- [Localization on RefCOCO+ (testA)](<https://www.sota2.com/research/sota/localization-on-refcoco-plus-testa.md>): Localization; RefCOCO+ (testA); Accuracy: 87.72.

- [Localization on RefCOCOg (val)](<https://www.sota2.com/research/sota/localization-on-refcocog-val.md>): Localization; RefCOCOg (val); Accuracy: 83.37.

- [Visual Question Answering on VQA v2 (Accuracy & Avg.)](<https://www.sota2.com/research/sota/visual-question-answering-on-vqa-v2-accuracy-and-avg.md>): Visual Question Answering; VQA v2; Accuracy: 76.7.

- [Science Question Answering on ScienceQA (EM)](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-em.md>): Science Question Answering; ScienceQA; Exact Match (EM): 84.33.

- [Multimodal Large Language Model Evaluation on MME](<https://www.sota2.com/research/sota/multimodal-large-language-model-evaluation-on-mme.md>): Multimodal Large Language Model Evaluation; MME; MME Score: 2260.

- [Multimodal Reasoning on AI2D (test)](<https://www.sota2.com/research/sota/multimodal-reasoning-on-ai2d-test.md>): Multimodal Reasoning; AI2D (test); Accuracy: 54.73.

- [Multi-view Spatiotemporal Reasoning on STSnu](<https://www.sota2.com/research/sota/multi-view-spatiotemporal-reasoning-on-stsnu.md>): Multi-view Spatiotemporal Reasoning; STSnu; Ego Accuracy: 38.7.

- [Image-Text Understanding on Flickr30k](<https://www.sota2.com/research/sota/image-text-understanding-on-flickr30k.md>): Image-Text Understanding; Flickr30K; Score: 89.33.

- [Document Visual Question Answering on DocVQA 30%](<https://www.sota2.com/research/sota/document-visual-question-answering-on-docvqa-30-percent.md>): Document Visual Question Answering; DocVQA 30%; KV Cache Size (MB): 189.74.

- [Visual Language Model Evaluation on MME](<https://www.sota2.com/research/sota/visual-language-model-evaluation-on-mme.md>): Visual Language Model Evaluation; MME; MME Score: 1697.

- [Video Understanding on SEED-Bench Video Understanding](<https://www.sota2.com/research/sota/video-understanding-on-seed-bench-video-understanding.md>): Video Understanding; SEED-Bench Video Understanding; Accuracy: 85.19.

- [Multimodal Question Answering and Reasoning on LMM Evaluation Suite (GQA, SQA-IMG, VQA-Text, POPE, MME, MMBench, MM-Vet) (test val)](<https://www.sota2.com/research/sota/multimodal-question-answering-and-reasoning-on-lmm-evaluation-suite-gqa-sqa-img-vqa-text-pope-mme-mmbench-mm-vet-test-val.md>): Multimodal Question Answering and Reasoning; LMM Evaluation Suite (GQA, SQA-IMG, VQA-Text, POPE, MME, MMBench, MM-Vet) (test val); GQA Score: 61.3.

- [Vision-Language Understanding on MMBench cn](<https://www.sota2.com/research/sota/vision-language-understanding-on-mmbench-cn.md>): Vision-Language Understanding; MMBench CN; Accuracy: 58.2.

- [Visual Grounding on BLINK](<https://www.sota2.com/research/sota/visual-grounding-on-blink.md>): Visual Grounding; BLINK; Accuracy: 54.71.

- [Localization on RefCOCO+ (testB)](<https://www.sota2.com/research/sota/localization-on-refcoco-plus-testb.md>): Localization; RefCOCO+ (testB); Accuracy: 74.13.

- [Visual Question Answering on GQA (GQA, Rel.)](<https://www.sota2.com/research/sota/visual-question-answering-on-gqa-gqa-rel.md>): Visual Question Answering; GQA; GQA Score: 60.

- [Visual Question Answering on MathVista](<https://www.sota2.com/research/sota/visual-question-answering-on-mathvista.md>): Visual Question Answering; MathVista; Total Accuracy: 55.2.

- [Multi-sensor Vision-Language Modeling on MAPLM](<https://www.sota2.com/research/sota/multi-sensor-vision-language-modeling-on-maplm.md>): Multi-sensor Vision-Language Modeling; MAPLM; FRM Score: 51.27.

- [Autonomous Driving Reasoning on DriveLMM-o1 NuScenes](<https://www.sota2.com/research/sota/autonomous-driving-reasoning-on-drivelmm-o1-nuscenes.md>): Reasoning; DriveLMM-o1 NuScenes; Risk Assessment Accuracy: 67.53.

- [Video Understanding on VideoMME](<https://www.sota2.com/research/sota/video-understanding-on-videomme.md>): Video Understanding; VideoMME; Overall Score: 85.19.

- [Science Question Answering on SQA IMG (Accuracy)](<https://www.sota2.com/research/sota/science-question-answering-on-sqa-img-accuracy.md>): Science Question Answering; SQA IMG; Accuracy: 69.3.

- [Real-world Visual Understanding on RealWorldQA (Score Metric)](<https://www.sota2.com/research/sota/real-world-visual-understanding-on-realworldqa-score-metric.md>): Real-world Visual Understanding; RealworldQA; Score: 68.6.

- [Aggregate Multimodal Evaluation on Aggregate](<https://www.sota2.com/research/sota/aggregate-multimodal-evaluation-on-aggregate.md>): Aggregate Multimodal Evaluation; Aggregate; Relative Performance: 97.6.

- [Autonomous Driving Reasoning on DriveLM](<https://www.sota2.com/research/sota/autonomous-driving-reasoning-on-drivelm.md>): Autonomous driving reasoning (cross-view risk object perception, action prediction, and planning); DriveLM; Accuracy: 78.

- [Visual Grounding on Visual Grounding Suite Average Performance](<https://www.sota2.com/research/sota/visual-grounding-on-visual-grounding-suite-average-performance.md>): Visual Grounding; Visual Grounding Suite Average Performance; Average Performance: 66.34.

- [Visual Question Answering on Text VQA (Accuracy, Avg., Retain %)](<https://www.sota2.com/research/sota/visual-question-answering-on-text-vqa-accuracy-avg-retain-percent.md>): Visual Question Answering; Text VQA; Accuracy: 76.96.

- [Text-focused Image Understanding on Text-focused Image Understanding Benchmarks (test)](<https://www.sota2.com/research/sota/text-focused-image-understanding-on-text-focused-image-understanding-benchmarks-test.md>): Text-focused Image Understanding; Text-focused image understanding benchmarks Chart, OCR, Doc, Info (test); Chart Score: 65.

- [Robotic Manipulation on Real-world environment](<https://www.sota2.com/research/sota/robotic-manipulation-on-real-world-environment.md>): Robotic Manipulation; Real-world environment; Bottle Handover Success Rate: 25.

- [Robotic Manipulation on RoboTwin simulation 1.0](<https://www.sota2.com/research/sota/robotic-manipulation-on-robotwin-simulation-1-0.md>): Robotic Manipulation; RoboTwin simulation 1.0; Hammer Success Rate: 71.

- [Object Hallucination Evaluation on POPE (Rand, Pop, Adv, Avg)](<https://www.sota2.com/research/sota/object-hallucination-evaluation-on-pope-rand-pop-adv-avg.md>): Object Hallucination Evaluation; POPE; Average Accuracy: 97.98.

- [Overall Performance on Aggregate 9 benchmarks](<https://www.sota2.com/research/sota/overall-performance-on-aggregate-9-benchmarks.md>): Overall Performance; Aggregate 9 benchmarks; Average Score: 62.

- [GUI Navigation on Mind2Web Cross-Task](<https://www.sota2.com/research/sota/gui-navigation-on-mind2web-cross-task.md>): GUI Navigation; Mind2Web Cross-Task; Element Accuracy: 57.09.

- [Localization on RefCOCO (val)](<https://www.sota2.com/research/sota/localization-on-refcoco-val.md>): Localization; RefCOCO (val); Accuracy: 86.15.

- [Localization on RefCOCO (testA)](<https://www.sota2.com/research/sota/localization-on-refcoco-testa.md>): Localization; RefCOCO (testA); Accuracy: 91.32.

- [Visual Question Answering on GQA (Accuracy, Avg.)](<https://www.sota2.com/research/sota/visual-question-answering-on-gqa-accuracy-avg.md>): Visual Question Answering; GQA; Accuracy: 58.9.

- [Long-form Video Multimodal Evaluation on Video-MME Long](<https://www.sota2.com/research/sota/long-form-video-multimodal-evaluation-on-video-mme-long.md>): Long-form Video Multimodal Evaluation; Video-MME Long; Video-MME-Long Score: 54.8.

- [Multimodal Reasoning on M3-Bench web](<https://www.sota2.com/research/sota/multimodal-reasoning-on-m3-bench-web.md>): Multimodal Reasoning; M3-Bench web; ME Score: 40.4.

- [Mathematical Reasoning on Math Vista](<https://www.sota2.com/research/sota/mathematical-reasoning-on-math-vista.md>): Mathematical Reasoning; Math Vista; Accuracy: 53.

- [Multimodal Reasoning on MMVet v1 (val)](<https://www.sota2.com/research/sota/multimodal-reasoning-on-mmvet-v1-val.md>): Multimodal Reasoning; MMVet v1 (val); Accuracy: 31.3.

- [Knowledge-based Visual Question Answering on OKVQA (test)](<https://www.sota2.com/research/sota/knowledge-based-visual-question-answering-on-okvqa-test.md>): Knowledge-based Visual Question Answering; OKVQA (test); Accuracy: 51.5.

- [Visual Question Answering on VQA v2 (VQAv2 Score and Average Metric)](<https://www.sota2.com/research/sota/visual-question-answering-on-vqa-v2-vqav2-score-and-average-metric.md>): Visual Question Answering; VQA v2; VQAv2 Score: 76.7.

- [Multi-skill Compositional Reasoning on MM-Vet](<https://www.sota2.com/research/sota/multi-skill-compositional-reasoning-on-mm-vet.md>): Multi-skill Compositional Reasoning; MM-Vet; Accuracy: 28.7.

- [Object Hallucination Evaluation on POPE (Accuracy and Rel. Acc)](<https://www.sota2.com/research/sota/object-hallucination-evaluation-on-pope-accuracy-and-rel-acc.md>): Object Hallucination Evaluation; POPE; Accuracy: 84.7.

- [Document Question Answering on DocVQA](<https://www.sota2.com/research/sota/document-question-answering-on-docvqa.md>): Document Question Answering; DocVQA; ANLS: 84.77.

- [Vision-Language Capability Evaluation on MME](<https://www.sota2.com/research/sota/vision-language-capability-evaluation-on-mme.md>): Vision-Language Capability Evaluation; MME; Score: 1564.

- [Localization on RefCOCO (testB)](<https://www.sota2.com/research/sota/localization-on-refcoco-testb.md>): Localization; RefCOCO (testB); Accuracy: 81.73.

- [Visual Grounding on V\*](<https://www.sota2.com/research/sota/visual-grounding-on-v.md>): Visual Grounding; V\*; Accuracy: 77.49.

- [Multimodal Visual Question Answering and Reasoning on LLaVA Evaluation Suite 1.5](<https://www.sota2.com/research/sota/multimodal-visual-question-answering-and-reasoning-on-llava-evaluation-suite-1-5.md>): Multimodal Visual Question Answering and Reasoning; LLaVA Evaluation Suite (GQA, SQA-IMG, VQA-Text, POPE, MME, MMBench-EN, MMBench-CN, MM-Vet) 1.5; GQA Accuracy: 57.9.

- [Visual Question Answering on GQA (GQA and RelAcc.)](<https://www.sota2.com/research/sota/visual-question-answering-on-gqa-gqa-and-relacc.md>): Visual Question Answering; GQA; GQA Score: 57.9.

- [Multimodal Reasoning Average on Combined (M3-Bench + Video-MME-Long)](<https://www.sota2.com/research/sota/multimodal-reasoning-average-on-combined-m3-bench-plus-video-mme-long.md>): Multimodal Reasoning Average; Combined (M3-Bench + Video-MME-Long); Average Score: 43.1.

- [Accuracy on Visual Question Answering on GQA](<https://www.sota2.com/research/sota/accuracy-on-visual-question-answering-on-gqa.md>): Visual Question Answering; GQA; Accuracy (GQA): 59.5.

- [Multimodal Understanding on MMBench-CN v1.1 (dev)](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmbench-cn-v1-1-dev.md>): Multimodal Understanding; MMBench-CN v1.1 (dev); Accuracy: 52.9.

- [Document Visual Question Answering on Doc VQA (Accuracy, Avg., Retain %)](<https://www.sota2.com/research/sota/document-visual-question-answering-on-doc-vqa-accuracy-avg-retain-percent.md>): Document Visual Question Answering; Doc VQA; Accuracy: 87.7.

- [Video Understanding on Vinoground](<https://www.sota2.com/research/sota/video-understanding-on-vinoground.md>): Video Understanding; Vinoground; Group Score: 12.6.

- [Visual Question Answering on GQA (GQA Score and Avg. Normalized Average Retention)](<https://www.sota2.com/research/sota/visual-question-answering-on-gqa-gqa-score-and-avg-normalized-average-retention.md>): Visual Question Answering; GQA; GQA Score: 61.7.

- [Visual Grounding on HRBench8K](<https://www.sota2.com/research/sota/visual-grounding-on-hrbench8k.md>): Visual Grounding; HRBench8K; Accuracy: 71.12.

- [Multimodal Understanding on Multimodal Evaluation Suite (GQA, MMBench, MMBench-CN, MME, POPE, SEED-Bench, TextVQA, VizWiz, OCRBench)](<https://www.sota2.com/research/sota/multimodal-understanding-on-multimodal-evaluation-suite-gqa-mmbench-mmbench-cn-mme-pope-seed-bench-textvqa-vizwiz-ocrbench.md>): Multimodal Understanding; Multimodal Evaluation Suite (GQA, MMBench, MMBench-CN, MME, POPE, SEED-Bench, TextVQA, VizWiz, OCRBench); GQA Score: 57.6.

- [Text-based Visual Question Answering on TextVQA (Accuracy and Latency)](<https://www.sota2.com/research/sota/text-based-visual-question-answering-on-textvqa-accuracy-and-latency.md>): Text-based Visual Question Answering; TextVQA; Average Score: 64.74.

- [Visual Question Answering on TextVQA (VQAText, Normalized Average Retention)](<https://www.sota2.com/research/sota/visual-question-answering-on-textvqa-vqatext-normalized-average-retention.md>): Visual Question Answering; TextVQA; VQAText Score: 58.7.

- [Visual Question Answering on GQA (GQA Score, Average, and Relative Performance %)](<https://www.sota2.com/research/sota/visual-question-answering-on-gqa-gqa-score-average-and-relative-performance-percent.md>): Visual Question Answering; GQA; GQA Score: 60.

- [Multimodal Understanding on MME v1.0 (val)](<https://www.sota2.com/research/sota/multimodal-understanding-on-mme-v1-0-val.md>): Multimodal Understanding; MME v1.0 (val); Score: 1344.

- [Object Hallucination Evaluation on POPE (test)](<https://www.sota2.com/research/sota/object-hallucination-evaluation-on-pope-test.md>): Object Hallucination Evaluation; POPE (test); Accuracy: 88.7.

- [Visual Question Answering on VQA v2 (VQAv2 and Average Score)](<https://www.sota2.com/research/sota/visual-question-answering-on-vqa-v2-vqav2-and-average-score.md>): Visual Question Answering; VQA v2; VQAv2 Accuracy: 78.2.

- [Visual Question Answering on VQA v2 (test val)](<https://www.sota2.com/research/sota/visual-question-answering-on-vqa-v2-test-val.md>): Visual Question Answering; VQA v2 (test val); Accuracy: 74.7.

- [Multimodal Evaluation on MME (Total Score & Retention)](<https://www.sota2.com/research/sota/multimodal-evaluation-on-mme-total-score-and-retention.md>): Multimodal Evaluation; MME; Total Score: 1710.

- [Multimodal Understanding and Question Answering on LLaVA-NeXT Evaluation Suite 13B (test)](<https://www.sota2.com/research/sota/multimodal-understanding-and-question-answering-on-llava-next-evaluation-suite-13b-test.md>): Multimodal Understanding and Question Answering; LLaVA-NeXT Evaluation Suite (GQA, SQA-IMG, VQA-Text, POPE, MME, MMB-EN, MMB-CN, MMVet) 13B (test); GQA Score: 62.7.

- [Multi-modal Understanding on MME (MME score)](<https://www.sota2.com/research/sota/multi-modal-understanding-on-mme-mme-score.md>): Multi-modal Understanding; MME; MME Score: 2174.

- [Visual Grounding on MMB v1.1](<https://www.sota2.com/research/sota/visual-grounding-on-mmb-v1-1.md>): Visual Grounding; MMB v1.1; Accuracy: 83.9.

- [Diagram Understanding on AI2D (EM)](<https://www.sota2.com/research/sota/diagram-understanding-on-ai2d-em.md>): Diagram Understanding; AI2D; Exact Match: 71.08.

- [Science Question Answering on ScienceQA (SQA and Retention Metrics)](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-sqa-and-retention-metrics.md>): Science Question Answering; ScienceQA; SQA Score: 68.4.

- [Image Captioning on Nocaps (CIDEr)](<https://www.sota2.com/research/sota/image-captioning-on-nocaps-cider.md>): Image Captioning; NoCaps; CIDEr: 1.11.

- [Multimodal Benchmarking on MMBench (MMB)](<https://www.sota2.com/research/sota/multimodal-benchmarking-on-mmbench-mmb.md>): Multimodal Benchmarking; MMBench (MMB); MMB Score: 63.7.

- [Multimodal Understanding on MMBench (MMB, Avg. Normalized Average Retention)](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmbench-mmb-avg-normalized-average-retention.md>): Multimodal Understanding; MMBench; MMB Score: 65.3.

- [Visual Grounding on MMStar](<https://www.sota2.com/research/sota/visual-grounding-on-mmstar.md>): Visual Grounding; MMStar; Accuracy: 65.13.

- [Multimodal Reasoning on MuirBench](<https://www.sota2.com/research/sota/multimodal-reasoning-on-muirbench.md>): Multimodal Reasoning; MuirBench; Accuracy: 47.1.

- [Visual Grounding on Multi-Dataset Suite (HRBench8K, VStarBench, RWQA, MMStar, GQA, MMBench v1.1, BLINK)](<https://www.sota2.com/research/sota/visual-grounding-on-multi-dataset-suite-hrbench8k-vstarbench-rwqa-mmstar-gqa-mmbench-v1-1-blink.md>): Visual Grounding; Visual Grounding benchmarks (HRBench8K, VStarBench, RWQA, MMStar, GQA, MMBench v1.1, BLINK); HRBench8K Score: 65.25.

- [Diagram Understanding on AI2D (Score and Avg. Rel.)](<https://www.sota2.com/research/sota/diagram-understanding-on-ai2d-score-and-avg-rel.md>): Diagram Understanding; AI2D; Score: 73.2.

- [Text-based Visual Question Answering on TextVQA VQAT (Accuracy)](<https://www.sota2.com/research/sota/text-based-visual-question-answering-on-textvqa-vqat-accuracy.md>): Text-based Visual Question Answering; TextVQA VQAT; Accuracy: 58.7.

- [Autonomous Driving Reasoning on DriveLMM-o1](<https://www.sota2.com/research/sota/autonomous-driving-reasoning-on-drivelmm-o1.md>): Autonomous Driving Reasoning; DriveLMM-o1; Risk Assessment: 71.5.

- [Text-based Visual Question Answering on TextVQA (test)](<https://www.sota2.com/research/sota/text-based-visual-question-answering-on-textvqa-test.md>): Text-based Visual Question Answering; TextVQA (test); Accuracy: 59.7.

- [Accuracy on Visual Question Answering on VQA v2](<https://www.sota2.com/research/sota/accuracy-on-visual-question-answering-on-vqa-v2.md>): Visual Question Answering; VQA v2; Accuracy: 78.7.

- [Multimodal Reasoning on M3-Bench robot](<https://www.sota2.com/research/sota/multimodal-reasoning-on-m3-bench-robot.md>): Multimodal Reasoning; M3-Bench robot; ME Score: 29.5.

- [Visual Question Answering on TextVQA (EM)](<https://www.sota2.com/research/sota/visual-question-answering-on-textvqa-em.md>): Visual Question Answering; TextVQA; Exact Match (EM): 71.68.

- [Multimodal Understanding on MMBench EN v1.1 (dev)](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmbench-en-v1-1-dev.md>): Multimodal Understanding; MMBench EN v1.1 (dev); Accuracy: 57.5.

- [Multimodal Reasoning on MMBench CN](<https://www.sota2.com/research/sota/multimodal-reasoning-on-mmbench-cn.md>): Multimodal Reasoning; MMBench CN; Accuracy: 79.

- [Visual Grounding on GQA](<https://www.sota2.com/research/sota/visual-grounding-on-gqa.md>): Visual Grounding; GQA; Accuracy: 52.66.

- [Multimodal Benchmarking on MMBench-CN](<https://www.sota2.com/research/sota/multimodal-benchmarking-on-mmbench-cn.md>): Multimodal Benchmarking; MMBench CN; Score: 72.25.

- [Science Question Answering on ScienceQA SQA-I](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-sqa-i.md>): Science Question Answering; ScienceQA SQA-I; Accuracy: 74.2.

- [Text-based Visual Question Answering on TextVQA (Score & Avg. Rel.)](<https://www.sota2.com/research/sota/text-based-visual-question-answering-on-textvqa-score-and-avg-rel.md>): Text-based Visual Question Answering; TextVQA; Score: 73.8.

- [Multimodal Understanding on MMStar (Score)](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmstar-score.md>): Multimodal Understanding; MMStar; Score: 49.7.

- [Visual Grounding on RWQA](<https://www.sota2.com/research/sota/visual-grounding-on-rwqa.md>): Visual Grounding; RWQA; Accuracy: 69.8.

- [Multimodal Benchmarking on MMBench EN](<https://www.sota2.com/research/sota/multimodal-benchmarking-on-mmbench-en.md>): Multimodal Benchmarking; MMBench EN; Accuracy (MMB): 68.6.

- [Chart Question Answering on ChartQA (Score & Retention Ratio R)](<https://www.sota2.com/research/sota/chart-question-answering-on-chartqa-score-and-retention-ratio-r.md>): Chart Question Answering; ChartQA; Score: 64.7.

- [Visual Reasoning on GQA (Accuracy)](<https://www.sota2.com/research/sota/visual-reasoning-on-gqa-accuracy.md>): Visual Reasoning; GQA; Accuracy: 61.7.

- [Multimodal Reasoning on MMBench (English and Chinese Scores)](<https://www.sota2.com/research/sota/multimodal-reasoning-on-mmbench-english-and-chinese-scores.md>): Multimodal Reasoning; MMBench; MMBench Accuracy (en): 63.7.

- [Image Captioning on NoCaps (test)](<https://www.sota2.com/research/sota/image-captioning-on-nocaps-test.md>): Image Captioning; NoCaps (test); CIDEr: 103.84.

- [Visual Question Answering on Text-Rich Evaluation Suite (OCR Bench, Text VQA, Chart QA, Doc VQA, Info VQA, Math Vista)](<https://www.sota2.com/research/sota/visual-question-answering-on-text-rich-evaluation-suite-ocr-bench-text-vqa-chart-qa-doc-vqa-info-vqa-math-vista.md>): Visual Question Answering; Text-Rich Evaluation Suite (OCR Bench, Text VQA, Chart QA, Doc VQA, Info VQA, Math Vista); Average Accuracy: 70.77.

- [Multimodal Question Answering on MMBench EN](<https://www.sota2.com/research/sota/multimodal-question-answering-on-mmbench-en.md>): Multimodal Question Answering; MMBench EN; Accuracy: 63.1.

- [Video Understanding on MLVU (Accuracy, Avg., Perf.)](<https://www.sota2.com/research/sota/video-understanding-on-mlvu-accuracy-avg-perf.md>): Video Understanding; MLVU; Accuracy: 55.4.

- [OCR-based Visual Question Answering on OCR-VQA](<https://www.sota2.com/research/sota/ocr-based-visual-question-answering-on-ocr-vqa.md>): OCR-based Visual Question Answering; OCR-VQA; Accuracy: 40.6.

- [Video Understanding on Video-MME (Short, Medium, Long, Overall, Avg., Perf.)](<https://www.sota2.com/research/sota/video-understanding-on-video-mme-short-medium-long-overall-avg-perf.md>): Video Understanding; Video-MME; Performance (Short): 54.4.

- [Multi-modal Benchmarking on SEED-Bench](<https://www.sota2.com/research/sota/multi-modal-benchmarking-on-seed-bench.md>): Multi-modal Benchmarking; SEED-Bench; Score: 51.5.

- [Multimodal Perception and Cognition on MME](<https://www.sota2.com/research/sota/multimodal-perception-and-cognition-on-mme.md>): Multimodal Perception and Cognition; MME; Overall Score: 2316.

- [Science Question Answering on ScienceQA (SQA)](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-sqa.md>): Science Question Answering; ScienceQA (SQA); Accuracy: 84.88.

- [Multimodal Understanding on MME (Score)](<https://www.sota2.com/research/sota/multimodal-understanding-on-mme-score.md>): Multimodal Understanding; MME; Score: 2295.

- [Web Navigation on Mind2Web Cross-Domain (EA, AF1, StepSR, TaskSR)](<https://www.sota2.com/research/sota/web-navigation-on-mind2web-cross-domain-ea-af1-stepsr-tasksr.md>): Web navigation; Mind2Web Cross-Domain; Element Accuracy (EA): 51.97.

- [Multimodal Benchmarking on MMB](<https://www.sota2.com/research/sota/multimodal-benchmarking-on-mmb.md>): Multimodal Benchmarking; MMB; Average Performance: 63.21.

- [Multimodal Reasoning on MMStar (Acc., AvgLen, Ratio)](<https://www.sota2.com/research/sota/multimodal-reasoning-on-mmstar-acc-avglen-ratio.md>): Multimodal Reasoning; MMStar; Accuracy: 56.8.

- [Visual Question Answering on OCRBench](<https://www.sota2.com/research/sota/visual-question-answering-on-ocrbench.md>): Visual Question Answering; OCRBench; Score: 712.

- [Document Parsing on OmniDocBench](<https://www.sota2.com/research/sota/document-parsing-on-omnidocbench.md>): Document Parsing; OmniDocBench; Overall Accuracy: 36.47.

- [Vision-Language Understanding on MMBench](<https://www.sota2.com/research/sota/vision-language-understanding-on-mmbench.md>): Vision-Language Understanding; MMBench; Accuracy: 65.3.

- [Visual Question Answering on AI2D (Exact Match)](<https://www.sota2.com/research/sota/visual-question-answering-on-ai2d-exact-match.md>): Visual Question Answering; AI2D; EM: 74.35.

- [Multi-modal Reasoning on MMVet](<https://www.sota2.com/research/sota/multi-modal-reasoning-on-mmvet.md>): Multi-modal Reasoning; MMVet; Score: 37.3.

- [Document Visual Question Answering on DocVQA (Accuracy)](<https://www.sota2.com/research/sota/document-visual-question-answering-on-docvqa-accuracy.md>): Document Visual Question Answering; DocVQA; Accuracy: 74.9.

- [Multimodal Evaluation on MME (score)](<https://www.sota2.com/research/sota/multimodal-evaluation-on-mme-score.md>): Multimodal Evaluation; MME; MME Score: 1856.

- [Image Captioning on COCO (test) (CIDEr)](<https://www.sota2.com/research/sota/image-captioning-on-coco-test-cider.md>): Image Captioning; COCO (test); CIDEr: 108.93.

- [Text-based Visual Question Answering on TextVQA (accuracy, relative accuracy)](<https://www.sota2.com/research/sota/text-based-visual-question-answering-on-textvqa-accuracy-relative-accuracy.md>): Text-based Visual Question Answering; TextVQA; Accuracy: 58.7.

- [Visual Question Answering on SciQA IMG](<https://www.sota2.com/research/sota/visual-question-answering-on-sciqa-img.md>): Visual Question Answering; SciQA-IMG; Accuracy: 69.1.

- [Comprehensive Multi-modal Evaluation on MME](<https://www.sota2.com/research/sota/comprehensive-multi-modal-evaluation-on-mme.md>): Comprehensive Multi-modal Evaluation; MME; Total Score: 1856.

- [Visual Question Answering on Vizwiz (val)](<https://www.sota2.com/research/sota/visual-question-answering-on-vizwiz-val.md>): Visual Question Answering; Vizwiz (val); VQA Score: 53.5.

- [Multimodal Evaluation on MMBench](<https://www.sota2.com/research/sota/multimodal-evaluation-on-mmbench.md>): Multimodal Evaluation; MMBench; MMB^CN Score: 63.3.

- [Visual Question Answering on TextVQA](<https://www.sota2.com/research/sota/visual-question-answering-on-textvqa.md>): Visual Question Answering; TextVQA; Accuracy: 82.1.

- [Multimodal Benchmarking on MMBench (Accuracy)](<https://www.sota2.com/research/sota/multimodal-benchmarking-on-mmbench-accuracy.md>): Multimodal Benchmarking; MMBench; Accuracy: 80.

- [Visual Question Answering on DocVQA](<https://www.sota2.com/research/sota/visual-question-answering-on-docvqa.md>): Visual Question Answering; DocVQA; Accuracy: 88.5.

- [Visual Question Answering on TextVQA (TextVQA Score)](<https://www.sota2.com/research/sota/visual-question-answering-on-textvqa-textvqa-score.md>): Visual Question Answering; TextVQA; TextVQA Score: 62.

- [Multimodal Reasoning on MMMU (test)](<https://www.sota2.com/research/sota/multimodal-reasoning-on-mmmu-test.md>): Multimodal Reasoning; MMMU (test); Accuracy: 26.7.

- [Science Question Answering on ScienceQA SQA-IMG](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa-sqa-img.md>): Science Question Answering; ScienceQA SQA-IMG; Accuracy: 71.1.

- [Visual Perception on MME Perception](<https://www.sota2.com/research/sota/visual-perception-on-mme-perception.md>): Visual Perception; MME Perception; MME^P: 1282.8.

- [Optical Character Recognition on OCRBench (accuracy)](<https://www.sota2.com/research/sota/optical-character-recognition-on-ocrbench-accuracy.md>): Optical Character Recognition Benchmarking; OCRBench; Accuracy: 75.5.

- [Object Hallucination Evaluation on POPE (Accuracy)](<https://www.sota2.com/research/sota/object-hallucination-evaluation-on-pope-accuracy.md>): Object Hallucination Evaluation; POPE; Accuracy (POPE): 85.7.

- [Visual Question Answering on MMBench-CN](<https://www.sota2.com/research/sota/visual-question-answering-on-mmbench-cn.md>): Visual Question Answering; MMBench CN; Accuracy: 61.

- [Multimodal Model Evaluation on MMBench Chinese](<https://www.sota2.com/research/sota/multimodal-model-evaluation-on-mmbench-chinese.md>): Multimodal Model Evaluation; MMBench Chinese; Accuracy: 62.2.

- [Multimodal Benchmarking on MMBench (Score and Relative Performance)](<https://www.sota2.com/research/sota/multimodal-benchmarking-on-mmbench-score-and-relative-performance.md>): Multimodal Benchmarking; MMBench; MMBench Score: 60.7.

- [Multimodal Understanding on LLaVA Evaluation Suite 1.5](<https://www.sota2.com/research/sota/multimodal-understanding-on-llava-evaluation-suite-1-5.md>): Multimodal Understanding; LLaVA Evaluation Suite 1.5; Average Score: 97.3.

- [Visual Question Answering on TextVQA (test)](<https://www.sota2.com/research/sota/visual-question-answering-on-textvqa-test.md>): Visual Question Answering; TextVQA (test); Accuracy: 58.

- [Multimodal Question Answering on MMBench CN](<https://www.sota2.com/research/sota/multimodal-question-answering-on-mmbench-cn.md>): Multimodal Question Answering; MMBench CN; Accuracy: 57.3.

- [Object Hallucination Evaluation on POPE](<https://www.sota2.com/research/sota/object-hallucination-evaluation-on-pope.md>): Object Hallucination Evaluation; POPE; Accuracy: 89.2.

- [Video Temporal Grounding on ActivityNet Grounding](<https://www.sota2.com/research/sota/video-temporal-grounding-on-activitynet-grounding.md>): Video Temporal Grounding; ActivityNet Grounding; R@0.3: 18.62.

- [Vision Understanding on MMBench](<https://www.sota2.com/research/sota/vision-understanding-on-mmbench.md>): Vision Understanding; MMBench; Accuracy: 65.3.

- [Multimodal Benchmarking on MMBench English](<https://www.sota2.com/research/sota/multimodal-benchmarking-on-mmbench-english.md>): Multimodal Benchmarking; MMBench English; Accuracy: 65.4.

- [Document Parsing on olmOCR-bench](<https://www.sota2.com/research/sota/document-parsing-on-olmocr-bench.md>): Document Parsing; olmOCR-bench; ArXiv Processing Accuracy: 1.2.

- [Visual Question Answering on GQA (GQA, Avg.)](<https://www.sota2.com/research/sota/visual-question-answering-on-gqa-gqa-avg.md>): Visual Question Answering; GQA; GQA Score: 60.9.

- [Visual Question Answering on TextVQA (TextVQA Metric)](<https://www.sota2.com/research/sota/visual-question-answering-on-textvqa-textvqa-metric.md>): Visual Question Answering; TextVQA; TextVQA Accuracy: 75.9.

- [Mathematical Visual Question Answering on MathVista](<https://www.sota2.com/research/sota/mathematical-visual-question-answering-on-mathvista.md>): Mathematical Visual Question Answering; MathVista; Accuracy: 51.9.

- [Multimodal Understanding on MMBench English](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmbench-english.md>): Multimodal Understanding; MMBench English; Accuracy: 64.9.

- [Text-based Visual Question Answering on TextVQA (Score)](<https://www.sota2.com/research/sota/text-based-visual-question-answering-on-textvqa-score.md>): Text-based Visual Question Answering; TextVQA; Score: 57.4.

- [Multimodal Evaluation on MME](<https://www.sota2.com/research/sota/multimodal-evaluation-on-mme.md>): Multimodal Evaluation; MME; Score: 2252.

- [Visual Question Answering on VQA v2 (Accuracy and Rel.)](<https://www.sota2.com/research/sota/visual-question-answering-on-vqa-v2-accuracy-and-rel.md>): Visual Question Answering; VQA v2; Accuracy: 78.3.

- [Information Visual Question Answering on InfoVQA](<https://www.sota2.com/research/sota/information-visual-question-answering-on-infovqa.md>): Information Visual Question Answering; InfoVQA; Accuracy: 69.49.

- [Text-based Visual Question Answering on VQAText](<https://www.sota2.com/research/sota/text-based-visual-question-answering-on-vqatext.md>): Text-based Visual Question Answering; VQAText; Accuracy: 57.02.

- [Visual Question Answering on GQA (test)](<https://www.sota2.com/research/sota/visual-question-answering-on-gqa-test.md>): Visual Question Answering; GQA (test); Accuracy: 60.8.

- [Visual Question Answering on GQA (Score)](<https://www.sota2.com/research/sota/visual-question-answering-on-gqa-score.md>): Visual Question Answering; GQA; Score: 60.

- [Knowledge-based Visual Question Answering on OKVQA](<https://www.sota2.com/research/sota/knowledge-based-visual-question-answering-on-okvqa.md>): Knowledge-based Visual Question Answering; OKVQA; Accuracy: 0.4037.

- [Multimodal Understanding on MMBench (MMB)](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmbench-mmb.md>): Multimodal Understanding; MMBench (MMB); Accuracy: 64.7.

- [Multimodal Evaluation on MM-Vet](<https://www.sota2.com/research/sota/multimodal-evaluation-on-mm-vet.md>): Multimodal Evaluation; MM-Vet; Score: 34.8.

- [Visual Question Answering on InfoVQA](<https://www.sota2.com/research/sota/visual-question-answering-on-infovqa.md>): Visual Question Answering; InfoVQA; Accuracy: 64.4.

- [Multimodal Evaluation on MMBench CN](<https://www.sota2.com/research/sota/multimodal-evaluation-on-mmbench-cn.md>): Multimodal Evaluation; MMBench CN; Accuracy: 56.

- [Visual Question Answering on MMBench](<https://www.sota2.com/research/sota/visual-question-answering-on-mmbench.md>): Visual Question Answering; MMBench (MMB); Accuracy: 65.5.

- [Multimodal Evaluation on MMStar](<https://www.sota2.com/research/sota/multimodal-evaluation-on-mmstar.md>): Multimodal Evaluation; MMStar; Accuracy: 52.9.

- [Multimodal Evaluation on MME (MME-P and MME-C)](<https://www.sota2.com/research/sota/multimodal-evaluation-on-mme-mme-p-and-mme-c.md>): Multimodal Evaluation; MME; MME-P Score: 1273.3.

- [Visual Question Answering on GQA (Accuracy)](<https://www.sota2.com/research/sota/visual-question-answering-on-gqa-accuracy.md>): Visual Question Answering; GQA; Accuracy: 61.7.

- [Visual Question Answering on VQA v2 (Performance Score)](<https://www.sota2.com/research/sota/visual-question-answering-on-vqa-v2-performance-score.md>): Visual Question Answering; VQA v2; Accuracy: 76.

- [Optical Character Recognition on OCRBench](<https://www.sota2.com/research/sota/optical-character-recognition-on-ocrbench.md>): Optical Character Recognition; OCRBench; Score: 686.

- [Video Understanding on LongVideoBench](<https://www.sota2.com/research/sota/video-understanding-on-longvideobench.md>): Video Understanding; LongVideoBench; Accuracy: 46.3.

- [Visual Question Answering on GQA (Mean accuracy)](<https://www.sota2.com/research/sota/visual-question-answering-on-gqa-mean-accuracy.md>): Visual Question Answering; GQA; Mean Accuracy: 57.9.

- [Diagram Understanding on AI2D (test)](<https://www.sota2.com/research/sota/diagram-understanding-on-ai2d-test.md>): Diagram Understanding; AI2D (test); Accuracy: 62.2.

- [Long Video Understanding on LongVideoBench (val)](<https://www.sota2.com/research/sota/long-video-understanding-on-longvideobench-val.md>): Long Video Understanding; LongVideoBench (val); Accuracy: 60.5.

- [Multimodal Reasoning on MMBench](<https://www.sota2.com/research/sota/multimodal-reasoning-on-mmbench.md>): Multimodal Reasoning; MMBench; Accuracy: 63.6.

- [Image Captioning on Flickr30k (test)](<https://www.sota2.com/research/sota/image-captioning-on-flickr30k-test.md>): Image Captioning; Flickr30k (test); CIDEr: 28.54.

- [Science Question Answering on ScienceQA](<https://www.sota2.com/research/sota/science-question-answering-on-scienceqa.md>): Science Question Answering; ScienceQA; Accuracy: 85.9.

- [Diagram Question Answering on AI2D](<https://www.sota2.com/research/sota/diagram-question-answering-on-ai2d.md>): Diagram Question Answering; AI2D; AI2D Accuracy: 80.3.

- [Multimodal Reasoning on MM-Vet](<https://www.sota2.com/research/sota/multimodal-reasoning-on-mm-vet.md>): Multimodal Reasoning; MM-Vet; MM-Vet Score: 44.3.

- [Multimodal Understanding on MMMU (val)](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmmu-val.md>): Multimodal Understanding; MMMU (val); MMMU Score: 36.6.

- [Visual Question Answering on VQAv2](<https://www.sota2.com/research/sota/visual-question-answering-on-vqav2.md>): Visual Question Answering; VQAv2; Accuracy: 75.7.

- [Document Visual Question Answering on DocVQA](<https://www.sota2.com/research/sota/document-visual-question-answering-on-docvqa.md>): Document Visual Question Answering; DocVQA; ANLS: 78.9.

- [Multimodal Understanding on MMBench](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmbench.md>): Multimodal Understanding; MMBench; Accuracy: 79.6.

- [Multimodal Model Evaluation on MMBench](<https://www.sota2.com/research/sota/multimodal-model-evaluation-on-mmbench.md>): Multimodal Model Evaluation; MMBench; Accuracy: 60.7.

- [Object Hallucination Evaluation on POPE (Accuracy Only)](<https://www.sota2.com/research/sota/object-hallucination-evaluation-on-pope-accuracy-only.md>): Object Hallucination Evaluation; POPE; Accuracy: 81.58.

- [Long Video Understanding on MLVU](<https://www.sota2.com/research/sota/long-video-understanding-on-mlvu.md>): Long Video Understanding; MLVU; Accuracy: 60.5.

- [Multimodal Understanding on MMBench CN](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmbench-cn.md>): Multimodal Understanding; MMBench CN; Accuracy: 56.7.

- [Text-based Visual Question Answering on TextVQA (Accuracy)](<https://www.sota2.com/research/sota/text-based-visual-question-answering-on-textvqa-accuracy.md>): Text-based Visual Question Answering; TextVQA; Accuracy: 72.2.

- [Visual Question Answering on ChartQA](<https://www.sota2.com/research/sota/visual-question-answering-on-chartqa.md>): Visual Question Answering; ChartQA; Accuracy: 78.12.

- [Hallucination Evaluation on POPE](<https://www.sota2.com/research/sota/hallucination-evaluation-on-pope.md>): Hallucination Evaluation; POPE; Accuracy: 80.2.

- [Visual Question Answering on VQA v2](<https://www.sota2.com/research/sota/visual-question-answering-on-vqa-v2.md>): Visual Question Answering; VQA v2; Accuracy: 79.47.

- [Visual Question Answering on VQA v2 (test-dev)](<https://www.sota2.com/research/sota/visual-question-answering-on-vqa-v2-test-dev.md>): Visual Question Answering; VQA v2 (test-dev); Overall Accuracy: 75.7.

- [Chart Question Answering on ChartQA](<https://www.sota2.com/research/sota/chart-question-answering-on-chartqa.md>): Chart Question Answering; ChartQA; Accuracy: 49.44.

- [Visual Question Answering on GQA](<https://www.sota2.com/research/sota/visual-question-answering-on-gqa.md>): Visual Question Answering; GQA; Accuracy: 61.7.

- [Video Understanding on MVBench](<https://www.sota2.com/research/sota/video-understanding-on-mvbench.md>): Video Understanding; MVBench; Accuracy: 46.2.

- [Multimodal Understanding on MMStar](<https://www.sota2.com/research/sota/multimodal-understanding-on-mmstar.md>): Multimodal Understanding; MMStar; Accuracy: not recorded.

- [Real-world Visual Question Answering on RealWorldQA](<https://www.sota2.com/research/sota/real-world-visual-question-answering-on-realworldqa.md>): Real-world Visual Question Answering; RealworldQA; Accuracy: not recorded.

- [Multi-modal Evaluation on MME (test)](<https://www.sota2.com/research/sota/multi-modal-evaluation-on-mme-test.md>): Multi-modal Evaluation; MME (test); Perception Score: not recorded.

- [Multimodal Reasoning on MMBench v1.1 (test)](<https://www.sota2.com/research/sota/multimodal-reasoning-on-mmbench-v1-1-test.md>): Multimodal Reasoning; MMBench v1.1 (test); Accuracy: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2502.11494>)

[PDF](<https://arxiv.org/pdf/2502.11494.pdf>)
