Vision-Language Understanding on MMBench
88.7AccuracyQwen3-VL-4B-Instruct
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen3-VL-4B-Instruct2026.02 | 88.7 | |
| Qwen2.5-VL-72B-InstructAttack=Benign2026.07 | 88.39 | |
| InternVL3.5-38BAttack=Benign2026.07 | 86.99 | |
| Optimization-based Token SelectionTarget Model=InternVL3.5-38B2026.07 | 86.22 | |
| Xiaomi-Robotics-0pre-training=with VL data2026.02 | 84.4 | |
| InternVL3.5-8BAttack=Benign2026.07 | 83.2 | |
| InternVL3.5-4BAttack=Benign2026.07 | 81.03 | |
| MolmoAct2026.02 | 80.1 | |
| Optimization-based Token SelectionTarget Model=InternVL3.5-8B2026.07 | 80.01 | |
| Qwen2.5-VL-7B-InstructAttack=Benign2026.07 | 79.87 | |
| Qwen2.5-VL-3B-InstructAttack=Benign2026.07 | 76.54 | |
| SmoothSMoE annealed (k=2)k=2, Smoothing mechanism=Annealed SmoothSMoE, Backbone=ViT-based, Model parameters=5.6B, Fine-tuning dataset=50% of the LLaVA-665K dataset, Number of experts=42026.06 | 71.13 | |
| SMoE (k=2)k=2, Smoothing mechanism=None, Backbone=ViT-based, Model parameters=5.6B, Fine-tuning dataset=50% of the LLaVA-665K dataset, Number of experts=42026.06 | 70.96 | |
| SmoothSMoE (k=2.5)k=2.5, Smoothing mechanism=SmoothSMoE, Backbone=ViT-based, Model parameters=5.6B, Fine-tuning dataset=50% of the LLaVA-665K dataset, Number of experts=42026.06 | 69.42 | |
| Upper BoundBackbone=LLaVA-v1.5-13B, Retained Tokens=Full, Pruning Ratio=0%2026.04 | 68.9 | |
| CLASPBackbone=LLaVA-v1.5-13B, Retained Tokens=192, Pruning Ratio=66.7%2026.04 | 68 | |
| CLASPBackbone=LLaVA-v1.5-13B, Retained Tokens=128, Pruning Ratio=77.8%2026.04 | 68 | |
| VanillaBackbone=LLaVA-NeXT-7B, Token Budget=2880 Tokens, Venue=-2026.02 | 67.4 | |
| VanillaBase Model=LLaVA-Next-7B, Token Reduction Rate=0.0%, N=28802025.05 | 67.4 | |
| SparseVLMBackbone=LLaVA-v1.5-13B, Retained Tokens=192, Pruning Ratio=66.7%2026.04 | 67.4 | |
| VanillaBackbone=LLaVA-NeXT-7B, Retained Tokens=2880, Compression Ratio=100%2026.04 | 67.4 | |
| PIO-FVLMBackbone=LLaVA-NeXT-7B, Token Budget=320 Tokens, Venue=Ours2026.02 | 66.2 | |
| MoBBase Model=LLaVA-Next-7B, Objectives=PA VP, Pruning budget K=320, Token Reduction Rate=88.9%, η-prior=true2025.05 | 65.8 | |
| SparseVLMBackbone=LLaVA-v1.5-13B, Retained Tokens=128, Pruning Ratio=77.8%2026.04 | 65.8 | |
| CDPrunerBackbone=LLaVA-NeXT-7B, Token Budget=320 Tokens, Venue=NeurIPS’252026.02 | 65.5 | |
| DARTBackbone=LLaVA-NeXT-7B, Token Budget=320 Tokens, Venue=EMNLP’252026.02 | 65.3 | |
| HoloVBackbone=LLaVA-NeXT-7B, Token Budget=320 Tokens, Venue=NeurIPS’252026.02 | 65.3 | |
| VanillaBase Model=LLaVA-1.5-7B, Token Reduction Rate=0.0%, N=5762025.05 | 64.7 | |
| CLASPBackbone=LLaVA-v1.5-13B, Retained Tokens=64, Pruning Ratio=88.9%2026.04 | 64.4 | |
| MoBBase Model=LLaVA-1.5-7B, Pruning budget K=192, Token Reduction Rate=66.7%, η-prior=true2025.05 | 64.1 | |
| TwigVLMBase Model=LLaVA-1.5-7B, Objectives=PA, Pruning budget K=192, Token Reduction Rate=66.7%2025.05 | 64 | |
| MoBBase Model=LLaVA-1.5-7B, Objectives=PA VP, Pruning budget K=192, Token Reduction Rate=66.7%, η-prior=false2025.05 | 63.8 | |
| DARTBase Model=LLaVA-1.5-7B, Objectives=VP, Pruning budget K=192, Token Reduction Rate=66.7%2025.05 | 63.6 | |
| MoBBase Model=LLaVA-1.5-7B, Pruning budget K=128, Token Reduction Rate=77.8%, η-prior=true2025.05 | 63.5 | |
| PyramidDropBase Model=LLaVA-1.5-7B, Objectives=PA, Pruning budget K=192, Token Reduction Rate=66.7%2025.05 | 63.3 | |
| VisionZipBackbone=LLaVA-NeXT-7B, Token Budget=320 Tokens, Venue=CVPR’252026.02 | 63.1 | |
| VisionZipBase Model=LLaVA-1.5-7B, Objectives=VP, Pruning budget K=192, Token Reduction Rate=66.7%2025.05 | 63 | |
| TokenCarveBase Model=LLaVA-1.5-7B, Objectives=PA VP, Pruning budget K=192, Token Reduction Rate=66.7%2025.05 | 63 | |
| HiREDBase Model=LLaVA-1.5-7B, Objectives=VP, Pruning budget K=192, Token Reduction Rate=66.7%2025.05 | 62.8 | |
| EvoCompBackbone=LLaVA-NeXT-7B, Retained Tokens=160, Compression Ratio=94.4%, l=02026.04 | 62.7 | |
| SparseVLMBase Model=LLaVA-1.5-7B, Objectives=PA, Pruning budget K=192, Token Reduction Rate=66.7%2025.05 | 62.5 | |
| FiCoCo-VBase Model=LLaVA-1.5-7B, Objectives=VP, Pruning budget K=192, Token Reduction Rate=66.7%2025.05 | 62.3 | |
| MustDropBase Model=LLaVA-1.5-7B, Objectives=PA VP, Pruning budget K=192, Token Reduction Rate=66.7%2025.05 | 62.3 | |
| MoBBase Model=LLaVA-1.5-7B, Pruning budget K=64, Token Reduction Rate=88.9%, η-prior=true2025.05 | 62.1 | |
| MustDropBackbone=LLaVA-NeXT-7B, Retained Tokens=160, Compression Ratio=94.4%2026.04 | 61.7 | |
| FastVBackbone=LLaVA-NeXT-7B, Token Budget=320 Tokens, Venue=ECCV’242026.02 | 61.6 | |
| SparseVLMBackbone=LLaVA-v1.5-13B, Retained Tokens=64, Pruning Ratio=88.9%2026.04 | 61.3 | |
| FastVBase Model=LLaVA-1.5-7B, Objectives=VP, Pruning budget K=192, Token Reduction Rate=66.7%2025.05 | 61.2 | |
| DARTBackbone=LLaVA-NeXT-7B, Retained Tokens=160, Compression Ratio=94.4%2026.04 | 61.2 | |
| SparseVLMBackbone=LLaVA-NeXT-7B, Retained Tokens=160, Compression Ratio=94.4%2026.04 | 61.1 | |
| CLASPBackbone=LLaVA-v1.5-13B, Retained Tokens=32, Pruning Ratio=94.4%2026.04 | 60.7 | |
| SparseVLMBackbone=LLaVA-NeXT-7B, Token Budget=320 Tokens, Venue=ICML’252026.02 | 60.6 | |
| ToMeBase Model=LLaVA-1.5-7B, Objectives=VP, Pruning budget K=192, Token Reduction Rate=66.7%2025.05 | 60.5 | |
| FastVBackbone=LLaVA-NeXT-7B, Retained Tokens=160, Compression Ratio=94.4%2026.04 | 60.3 | |
| VisionZipBackbone=LLaVA-NeXT-7B, Retained Tokens=160, Compression Ratio=94.4%2026.04 | 60.1 | |
| PyramidDropBackbone=LLaVA-NeXT-7B, Retained Tokens=160, Compression Ratio=94.4%2026.04 | 60 | |
| LLaVA-PruMergeBase Model=LLaVA-1.5-7B, Objectives=VP, Pruning budget K=192, Token Reduction Rate=66.7%2025.05 | 59.6 | |
| GlobalCom^2Backbone=LLaVA-NeXT-7B, Retained Tokens=160, Compression Ratio=94.4%2026.04 | 59.1 | |
| MolmoE-1B-7BNA=1.58B, Avgk=8.002026.06 | 58.24 | |
| AdaMoENA=1.39B, Avgk=6.132026.06 | 56.95 | |
| GeMoENA=1.32B, Avgk=5.432026.06 | 56.7 | |
| SparseVLMBackbone=LLaVA-v1.5-13B, Retained Tokens=32, Pruning Ratio=94.4%2026.04 | 56.2 | |
| DARTBackbone=LLaVA-NeXT-7B, Retained Tokens=160, Compression Ratio=94.4%, l=02026.04 | 56.2 | |
| Top-pNA=2.05B, Avgk=12.652026.06 | 55.75 | |
| MMaDA+VRCDL=192, FR=0.252026.05 | 54.84 | |
| MMaDA+VRCDL=384, FR=0.252026.05 | 53.52 | |
| MMaDAL=192, FR=0.252026.05 | 53.45 | |
| MMaDA+VRCDL=192, FR=0.1252026.05 | 53.45 | |
| MMaDA+VRCDL=192, FR=0.52026.05 | 52.87 | |
| MMaDA+VRCDL=384, FR=0.1252026.05 | 52.81 | |
| MMaDAL=192, FR=0.1252026.05 | 52.09 | |
| GKL-KDCLIP Model=GKL-KD2025.03 | 52.06 | |
| KL-KDCLIP Model=KL-KD [27]2025.03 | 51.89 | |
| DYNMoENA=1.40B, Avgk=6.192026.06 | 51.72 | |
| MMaDA+VRCDL=384, FR=0.52026.05 | 51.32 | |
| MoE++NA=1.48B, Avgk=7.032026.06 | 51.2 | |
| MMaDAL=384, FR=0.1252026.05 | 50.9 | |
| MMaDAL=384, FR=0.252026.05 | 50.82 | |
| MMaDAL=192, FR=0.52026.05 | 50.66 | |
| BaselineCLIP Model=Baseline2025.03 | 49.66 | |
| MMaDAL=384, FR=0.52026.05 | 44.65 | |
| Optimization-based Token SelectionTarget Model=InternVL3.5-4B2026.07 | 25 | |
| pi_0.52026.02 | 22.1 | |
| Optimization-based Token SelectionTarget Model=Qwen2.5-VL-72B-Instruct2026.07 | 0.08 | |
| pi_02026.02 | 0 | |
| Xiaomi-Robotics-0pre-training=without VL data2026.02 | 0 | |
| Optimization-based Token SelectionTarget Model=Qwen2.5-VL-3B-Instruct2026.07 | 0 | |
| Optimization-based Token SelectionTarget Model=Qwen2.5-VL-7B-Instruct2026.07 | 0 |