Visual Question Answering on MMBench
92.1AccuracyDDP
Evaluation Results
| Method | Links | |
|---|---|---|
| DDPbackbone=Gemini-3-Pro2026.04 | 92.1 | |
| Gemini 2.5 Pro2026.04 | 90.1 | |
| Gemini-3-pro2026.04 | 88.4 | |
| OpenVLThinkerV2Parameters=8B2026.04 | 88.2 | |
| Vision-G12026.04 | 88 | |
| VL-Rethinker-7BParameters=7B2026.04 | 87.9 | |
| OpenVLThinker-7BParameters=7B2026.04 | 87.8 | |
| MM-Eureka-7BParameters=7B2026.04 | 87.7 | |
| Qwen3-VL-2B-Instruct + MoDA (ours)LLM=Qwen3-2B2025.06 | 87.4 | |
| OVR-7BParameters=7B2026.04 | 86.6 | |
| OneThinker-8BParameters=8B2026.04 | 86.6 | |
| Qwen3-VL-2B-InstructLLM=Qwen3-2B2025.06 | 86.5 | |
| Qwen3-VL GDPORL Strategy=GDPO2026.04 | 86.1 | |
| Qwen3-VL GRPORL Strategy=GRPO2026.04 | 86 | |
| Qwen3-VL-Instruct-8BParameters=8B2026.04 | 85.1 | |
| GPT-4o2026.04 | 84.3 | |
| VanillaSource=Qwen’25, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=0%2026.06 | 84 | |
| MiMo-VL-7B +PCBackbone Model=MiMo-VL-7B, Training Paradigm (+PC)=Yes2026.03 | 83.8 | |
| VanillaSource=Qwen’25, Token Reduction=0%, Backbone=Qwen2.5-VL-7B2026.06 | 83.7 | |
| GPT-4o2026.04 | 83.4 | |
| MiMo-VL-7BBackbone Model=MiMo-VL-7B, Training Paradigm (+PC)=No2026.03 | 83.2 | |
| Qwen2.5-VL-7B +PCBackbone Model=Qwen2.5-VL-7B, Training Paradigm (+PC)=Yes2026.03 | 82.5 | |
| InternVL-1.5Model Size=1.5-26B, Access Type=Open-source2024.07 | 82.3 | |
| Gemini-1.5-pro2026.04 | 82.3 | |
| IXC-2.5-7BModel Size=7B2024.07 | 82.2 | |
| Qwen2.5-VL-7BBackbone Model=Qwen2.5-VL-7B, Training Paradigm (+PC)=No2026.03 | 82.2 | |
| GPT-4VAccess Type=Closed-source API2024.07 | 81.3 | |
| PDrop+RerouteSource=Ours, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=66.7%2026.06 | 80 | |
| IDPrunerRetained Tokens=25%2026.02 | 79.73 | |
| SCOPERetained Tokens=25%2026.02 | 79.38 | |
| Qwen2.5-VL-3BBackbone Model=Qwen2.5-VL-3B, Training Paradigm (+PC)=No2026.03 | 79.1 | |
| Qwen2.5-VL-3B +PCBackbone Model=Qwen2.5-VL-3B, Training Paradigm (+PC)=Yes2026.03 | 79.1 | |
| PDrop+RerouteSource=Ours, Token Reduction=66.7%, Backbone=Qwen2.5-VL-7B2026.06 | 79 | |
| PDropSource=CVPR’25, Token Reduction=66.7%, Backbone=Qwen2.5-VL-7B2026.06 | 78.6 | |
| SFT + VIGILBase Model=Qwen2.5-VL-72B2026.06 | 78.5 | |
| FastVSource=ECCV’24, Token Reduction=66.7%, Backbone=Qwen2.5-VL-7B2026.06 | 77.8 | |
| SFT + DA-DPOBase Model=Qwen2.5-VL-72B2026.06 | 77.8 | |
| SFT + SimPOBase Model=Qwen2.5-VL-72B2026.06 | 77.5 | |
| FastV+RerouteSource=Ours, Token Reduction=66.7%, Backbone=Qwen2.5-VL-7B2026.06 | 77.3 | |
| SFT + DPOBase Model=Qwen2.5-VL-72B2026.06 | 77.2 | |
| FastV+RerouteSource=Ours, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=66.7%2026.06 | 76.9 | |
| SFT onlyBase Model=Qwen2.5-VL-72B2026.06 | 76.8 | |
| SFT + HA-DPOBase Model=Qwen2.5-VL-72B2026.06 | 76.5 | |
| PDrop+RerouteSource=Ours, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=77.8%2026.06 | 75.6 | |
| PDrop+RerouteSource=Ours, Token Reduction=77.8%, Backbone=Qwen2.5-VL-7B2026.06 | 75.4 | |
| PDropSource=CVPR’25, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=66.7%2026.06 | 74.2 | |
| Gemini-ProAccess Type=Closed-source API2024.07 | 73.9 | |
| TinyLLaVA†LLM=Qwen2.5-3B2026.07 | 73.1 | |
| Imp-3BLLM=Phi2-2.7B2026.07 | 72.9 | |
| PDropSource=CVPR’25, Token Reduction=77.8%, Backbone=Qwen2.5-VL-7B2026.06 | 72.8 | |
| FastV+RerouteSource=Ours, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=77.8%2026.06 | 72.6 | |
| SFT + VIGILBase Model=Qwen2.5-VL-7B2026.06 | 72.5 | |
| LLaVA-MoRE SigLIP-S2 + MoDA (ours)LLM=LLaMA 3.1-8B2025.06 | 72.4 | |
| LLaVA-MoRE OpenAI CLIPLLM=LLaMA 3.1-8B2025.06 | 72.3 | |
| LLaVA-MoRE OpenAI CLIP + MoDA (ours)LLM=LLaMA 3.1-8B2025.06 | 72 | |
| SFT + DA-DPOBase Model=Qwen2.5-VL-7B2026.06 | 72 | |
| LLaVA-MoRE SigLIP-S2LLM=LLaMA 3.1-8B2025.06 | 71.8 | |
| SFT + SimPOBase Model=Qwen2.5-VL-7B2026.06 | 71.5 | |
| SFT + DPOBase Model=Qwen2.5-VL-7B2026.06 | 71.2 | |
| LLaVA-KDLLM=Qwen2.5-1.5B2026.07 | 71 | |
| PDrop+RerouteSource=Ours, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=88.9%2026.06 | 70.8 | |
| TRAGLLM=Qwen2.5-1.5B2026.07 | 70.8 | |
| TinyLLaVA†LLM=Qwen2.5-1.5B2026.07 | 70.6 | |
| SFT onlyBase Model=Qwen2.5-VL-7B2026.06 | 70.5 | |
| FastVSource=ECCV’24, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=66.7%2026.06 | 69.9 | |
| LLaVA-CapLLM=Vicuna-13B, #PT=558K, #IT=665K, Representation=E + T2024.03 | 69.4 | |
| LLaVA-VTLLM=Vicuna-13B, #PT=558K, #IT=665K, Representation=E + T2024.03 | 69.4 | |
| LLaVA-SGLLM=Vicuna-13B, #PT=558K, #IT=665K, Representation=E + T2024.03 | 69.2 | |
| LLaVA-DCapLLM=Vicuna-13B, #PT=558K, #IT=665K, Representation=E + T2024.03 | 68.9 | |
| LLaVA-MoDLLM=Qwen2-1.5B2026.07 | 68.9 | |
| ShareGPT-4VLLM=Vicuna-7B2025.06 | 68.8 | |
| BunnyLLM=Phi2-2.7B2026.07 | 68.6 | |
| FastV+RerouteSource=Ours, Token Reduction=77.8%, Backbone=Qwen2.5-VL-7B2026.06 | 68.1 | |
| MoE-LLaVALLM=Phi2-2.7B2026.07 | 68 | |
| LLaVA-1.5-13B + QLIPModel=LLaVA-1.5-13B, Cropped Images=true2025.05 | 67.9 | |
| LLaVA-1.5 (V-13B)LLM=Vicuna-13B, #PT=558K, #IT=665K, Representation=Visual Embeddings (E)2024.03 | 67.7 | |
| VanillaRetained Tokens=576, Compression Ratio=100%2026.04 | 67.7 | |
| LLaVA-1.5LLM=Vicuna-13B2025.06 | 67.7 | |
| MMFuserBackbone=LLaVA-1.5-7B2026.02 | 67.5 | |
| TinyLLaVA†LLM=Qwen1.5-4B2026.07 | 67.5 | |
| LLaVA-1.5-13BModel=LLaVA-1.5-13B, Cropped Images=false2025.05 | 67.4 | |
| PDropSource=CVPR’25, Token Reduction=88.9%, Backbone=Qwen2.5-VL-7B2026.06 | 67.4 | |
| TinyLLaVALLM=Phi2-2.7B2026.07 | 66.9 | |
| Dense ConnectorBackbone=LLaVA-1.5-7B2026.02 | 66.8 | |
| EvoComp (l = 0)Retained Tokens=64, Compression Ratio=88.9%, Compressor layers (l)=0, Transfer setting=from LLaVA-1.5-7B to LLaVA-1.5-13B2026.04 | 66.8 | |
| TRAGLLM=Qwen1.5-1.8B2026.07 | 66.6 | |
| FastVSource=ECCV’24, Token Reduction=77.8%, Backbone=Qwen2.5-VL-7B2026.06 | 66.5 | |
| TGIFBackbone=LLaVA-1.5-7B2026.02 | 66.4 | |
| LLaVA-1.5-7BToken Retention Rate=576, Base Model=LLaVA-1.5-7B2025.06 | 66.31 | |
| PDrop+RerouteSource=Ours, Token Reduction=88.9%, Backbone=Qwen2.5-VL-7B2026.06 | 66 | |
| KD†LLM=Qwen1.5-1.8B2026.07 | 65.8 | |
| PDropSource=CVPR’25, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=77.8%2026.06 | 65.7 | |
| DARTRetained Tokens=64, Compression Ratio=88.9%2026.04 | 65.5 | |
| EvoComp (l = 2)Retained Tokens=64, Compression Ratio=88.9%, Compressor layers (l)=2, Transfer setting=from LLaVA-1.5-7B to LLaVA-1.5-13B2026.04 | 65.5 | |
| SparseVLM-v1.5Token Retention Rate=192, Base Model=LLaVA-1.5-7B2025.06 | 65.41 | |
| SparseVLM-v1.5Token Retention Rate=128, Base Model=LLaVA-1.5-7B2025.06 | 65.25 | |
| VisionDropToken Retention Rate=192, Base Model=LLaVA-1.5-7B2025.06 | 65.19 | |
| VisPrunerToken Retention Rate=192, Base Model=LLaVA-1.5-7B2025.06 | 64.91 | |
| VisionZipRetained Tokens=64, Compression Ratio=88.9%2026.04 | 64.9 | |
| LLaVA-1.5 + MoDA (ours)LLM=Vicuna-7B2025.06 | 64.8 |