Visual Question Answering on GQA (GQA, Avg.)
64.83GQA ScoreFull fine-tuning
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Full fine-tuningBackbone=LLaVA-1.5-13B2026.03 | 64.83 | — | |
| Latent DenoisingArchitecture=Qwen-2.5-VL2026.04 | 64.6 | — | |
| Latent DenoisingArchitecture=LLaVA+SigLIP2026.04 | 64.3 | — | |
| VanillaToken Budget=2880, Retention Ratio=100%2026.05 | 64.2 | 100 | |
| LoRABackbone=LLaVA-1.5-7B2026.03 | 63.98 | — | |
| BitFitBackbone=LLaVA-1.5-13B2026.03 | 63.98 | — | |
| Full fine-tuningBackbone=LLaVA-1.5-7B2026.03 | 63.92 | — | |
| CoVFTBackbone=LLaVA-1.5-13B2026.03 | 63.84 | — | |
| FreezeBackbone=LLaVA-1.5-13B2026.03 | 63.73 | — | |
| Latent DenoisingArchitecture=LLaVA+CLIP2026.04 | 63.5 | — | |
| BitFitBackbone=LLaVA-1.5-7B2026.03 | 63.43 | — | |
| VQRAELLM=Vicuna-13B, Token Type=2D-Continuous, Res.=2562026.05 | 63.4 | — | |
| CoVFTBackbone=LLaVA-1.5-7B2026.03 | 63.37 | — | |
| LLaVA-1.5Backbone=LLaVA-1.5-13B2026.03 | 63.3 | — | |
| LLaVA-1.5LLM=LLaMA-13B [59], Model Type=Dense Model2025.07 | 63.3 | 64.2 | |
| LLaVA-1.5-13BToken=576, Backbone=LLaVA-1.5-13B2026.04 | 63.3 | — | |
| ShareGPT4VLLM=Vicuna-7B, Token Type=2D-Continuous, Res.=3362026.05 | 63.3 | — | |
| SVPTBackbone=LLaVA-1.5-7B2026.03 | 63.26 | — | |
| LRCPToken Budget=640, Retention Ratio=↓ 77.8%2026.05 | 63.2 | 98.3 | |
| VPTBackbone=LLaVA-1.5-7B2026.03 | 63.17 | — | |
| Show-o2# Params.=7B2026.06 | 63.1 | — | |
| FreezeBackbone=LLaVA-1.5-7B2026.03 | 63.07 | — | |
| VanillaBackbone=InternVL2-8B, Retention Rate=100%2026.05 | 63 | 100 | |
| LightKVBackbone=InternVL2-8B, Retention Rate=60%2026.05 | 63 | 100.19 | |
| ApETToken Budget=640, Retention Ratio=↓ 77.8%2026.05 | 63 | 97.6 | |
| TokenFlow-XL*# Params.=14B2026.06 | 62.5 | — | |
| VanillaRetained Tokens=576, Token Reduction Ratio=100%2026.03 | 62.4 | 100 | |
| WinTokLLM=Qwen3-8B, Token Type=1D-Continuous, Res.=2562026.05 | 62.4 | — | |
| VLM-UniDDT# Params.=4B+1B2026.06 | 62.3 | — | |
| Our MethodLLM=Phi2-2.7B [62], Model Type=Sparse Model, Expert Selection (Top-k)=Top42025.07 | 62.2 | 62.3 | |
| BaselineArchitecture=LLaVA+SigLIP2026.04 | 62.1 | — | |
| LLaVA-1.5Backbone=LLaVA-1.5-7B2026.03 | 62 | — | |
| LLaVA-1.5-7BVision Token=5762026.03 | 62 | — | |
| QMoPVision Token=1442026.03 | 62 | — | |
| LLaVA-1.5LLM=Vicuna-7B [60], Model Type=Dense Model2025.07 | 62 | 61.2 | |
| PDropPruning Mode=T, Pruning Ratio=0.46, Backbone=LLaVA-v1.5-7B, Supervised fine-tuning=true, TFLOPs=3.092026.04 | 62 | — | |
| ToFuBackbone=InternVL2-8B, Retention Rate=60%2026.05 | 62 | 95.49 | |
| ToMe (P)Backbone=InternVL2-8B, Retention Rate=60%2026.05 | 62 | 97.86 | |
| ToMe (P)Backbone=InternVL2-8B, Retention Rate=55%2026.05 | 62 | 95.62 | |
| ToFuBackbone=InternVL2-8B, Retention Rate=55%2026.05 | 62 | 95.82 | |
| LightKVBackbone=InternVL2-8B, Retention Rate=55%2026.05 | 62 | 99.58 | |
| LLaVA-v1.5LLM=Vicuna-7B, Token Type=2D-Continuous, Res.=3362026.05 | 62 | — | |
| Janus-ProLLM=DeepSeek-LLM-7B, Token Type=2D-Continuous, Res.=3842026.05 | 62 | — | |
| LLaVA-v1.5# Params.=7B2026.06 | 62 | — | |
| Janus-Pro# Params.=7B2026.06 | 62 | — | |
| LLaVA1.5-7BPruning budget (keep tokens)=Full (No pruning), Base model=LLaVA1.5-7B2026.04 | 61.97 | — | |
| UncompressedPruning Mode=/, Pruning Ratio=/, Backbone=LLaVA-v1.5-7B, Supervised fine-tuning=false, TFLOPs=5.632026.04 | 61.9 | — | |
| CoMPPruning Mode=C, Pruning Ratio=0.46, Backbone=LLaVA-v1.5-7B, Supervised fine-tuning=true, TFLOPs=2.942026.04 | 61.9 | — | |
| BaselineArchitecture=LLaVA+CLIP2026.04 | 61.9 | — | |
| MoE-LLaVA-4Top2LLM=Phi2-2.7B [62], Model Type=Sparse Model, Expert Selection (Top-k)=4Top22025.07 | 61.4 | 61.1 | |
| VisionZipToken Budget=640, Retention Ratio=↓ 77.8%2026.05 | 61.3 | 97.4 | |
| PRUNESIDRetained Tokens=192, Token Reduction Ratio=↓ 66.7%2026.03 | 61.2 | 98.7 | |
| Our MethodLLM=StableLM-1.6B [65], Model Type=Sparse Model, Expert Selection (Top-k)=Top42025.07 | 61.1 | 58.8 | |
| LRCPToken Budget=320, Retention Ratio=↓ 88.9%2026.05 | 61.1 | 95.3 | |
| UniTokLLM=LLaMA2-7B, Token Type=2D-Discrete, Res.=2562026.05 | 61.1 | — | |
| Liquid# Params.=8B2026.06 | 61.1 | — | |
| PiToMeBackbone=InternVL2-8B, Retention Rate=55%2026.05 | 61 | 95.54 | |
| ApETToken Budget=320, Retention Ratio=↓ 88.9%2026.05 | 61 | 94.2 | |
| DARTPruning Mode=T, Pruning Ratio=0.46, Backbone=LLaVA-v1.5-7B, Supervised fine-tuning=true, TFLOPs=3.182026.04 | 60.9 | — | |
| Mogao# Params.=7B2026.06 | 60.9 | — | |
| VILA-U# Params.=7B2026.06 | 60.8 | — | |
| FitPruneVision Token=1922026.03 | 60.4 | — | |
| VanillaModel=Qwen2.5-VL-7B-Instruct, Vision token retention rate=55%2026.05 | 60.4 | 100 | |
| VisionZipRetained Tokens=192, Token Reduction Ratio=↓ 66.7%2026.03 | 60.3 | 98.3 | |
| MoE-LLaVA-4Top2LLM=StableLM-1.6B [65], Model Type=Sparse Model, Expert Selection (Top-k)=4Top22025.07 | 60.3 | 57.6 | |
| SparseVLMToken Budget=640, Retention Ratio=↓ 77.8%2026.05 | 60.3 | 95.7 | |
| Emu3LLM=8B (from scratch), Token Type=2D-Discrete, Res.=5122026.05 | 60.3 | — | |
| TokenFlow-LLLM=Vicuna-13B, Token Type=2D-Discrete, Res.=2562026.05 | 60.3 | — | |
| SemHiTokLLM=Qwen2.5-7B-Instruct, Token Type=2D-Discrete, Res.=2562026.05 | 60.3 | — | |
| JanusFlow# Params.=1.5B2026.06 | 60.3 | — | |
| Emu3# Params.=8B2026.06 | 60.3 | — | |
| RCPToken=192, Backbone=LLaVA-1.5-13B2026.04 | 60.2 | — | |
| BaselineArchitecture=Qwen-2.5-VL2026.04 | 60.2 | — | |
| PRUNESIDRetained Tokens=128, Token Reduction Ratio=↓ 77.8%2026.03 | 60.1 | 97.4 | |
| VisionZipPruning Mode=T, Pruning Ratio=0.46, Backbone=LLaVA-v1.5-7B, Supervised fine-tuning=true, TFLOPs=3.012026.04 | 60.1 | — | |
| DARTVision Token=1922026.03 | 60 | — | |
| DARTPruning Mode=T, Pruning Ratio=0.46, Backbone=LLaVA-v1.5-7B, Supervised fine-tuning=false, TFLOPs=3.182026.04 | 60 | — | |
| DivPrunePruning Mode=T, Pruning Ratio=0.46, Backbone=LLaVA-v1.5-7B, Supervised fine-tuning=false, TFLOPs=3.012026.04 | 60 | — | |
| CoMPPruning Mode=C, Pruning Ratio=0.62, Backbone=LLaVA-v1.5-7B, Supervised fine-tuning=true, TFLOPs=2.072026.04 | 60 | — | |
| PiToMeBackbone=InternVL2-8B, Retention Rate=60%2026.05 | 60 | 95.99 | |
| SparseVLMPruning Mode=T, Pruning Ratio=0.46, Backbone=LLaVA-v1.5-7B, Supervised fine-tuning=false, TFLOPs=3.012026.04 | 59.5 | — | |
| Qwen-VLLLM=Qwen-7B [4], Model Type=Dense Model2025.07 | 59.3 | — | |
| VisionZipPruning Mode=T, Pruning Ratio=0.46, Backbone=LLaVA-v1.5-7B, Supervised fine-tuning=false, TFLOPs=3.012026.04 | 59.3 | — | |
| SparseVLMToken Budget=320, Retention Ratio=↓ 88.9%2026.05 | 59.3 | 93.7 | |
| VisionZipToken Budget=320, Retention Ratio=↓ 88.9%2026.05 | 59.3 | 93.6 | |
| Janus-Pro# Params.=1.5B2026.06 | 59.3 | — | |
| ToMeModel=Qwen2.5-VL-7B-Instruct, Vision token retention rate=55%2026.05 | 59.1 | 100.04 | |
| LightKVModel=Qwen2.5-VL-7B-Instruct, Vision token retention rate=55%2026.05 | 59.1 | 101.37 | |
| MobileVLMLLM=MobileLLaMA-2.7B [63], Model Type=Dense Model2025.07 | 59 | — | |
| VisionZipRetained Tokens=128, Token Reduction Ratio=↓ 77.8%2026.03 | 58.7 | 96.2 | |
| HiREDVision Token=1922026.03 | 58.7 | — | |
| FastVModel=Qwen2.5-VL-7B-Instruct, Vision token retention rate=55%2026.05 | 58.7 | 98.77 | |
| ToFuModel=Qwen2.5-VL-7B-Instruct, Vision token retention rate=55%2026.05 | 58.7 | 100.75 | |
| LRCPToken Budget=160, Retention Ratio=↓ 94.4%2026.05 | 58.6 | 91.8 | |
| WinTokLLM=Qwen3-8B, Token Type=1D-Continuous, Res.=256, semantic tokens=642026.05 | 58.5 | — | |
| PiToMeModel=Qwen2.5-VL-7B-Instruct, Vision token retention rate=55%2026.05 | 58.4 | 100.24 | |
| ApETToken Budget=160, Retention Ratio=↓ 94.4%2026.05 | 58.4 | 90.1 | |
| LiquidLLM=Gemma-7B, Token Type=2D-Discrete, Res.=5122026.05 | 58.4 | — | |
| RCPToken=128, Backbone=LLaVA-1.5-13B2026.04 | 58.31 | — | |
| PRUNESIDRetained Tokens=64, Token Reduction Ratio=↓ 88.9%2026.03 | 58.3 | 94.4 |