Visual Question Answering on SQA
93.5AccuracyVanilla
Evaluation results
| Method | Links | ||
|---|---|---|---|
| VanillaSource=Qwen’25, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=0%2026.06 | 93.5 | — | |
| VanillaBackbone=Qwen3-VL-4B, Retention Ratio=100%2026.04 | 93.42 | 100 | |
| VisionZip w/ DSTPBackbone=Qwen3-VL-4B, Retention Ratio=33.3%2026.04 | 92.08 | 97.4 | |
| FastV w/ DSTPBackbone=Qwen3-VL-4B, Retention Ratio=33.3%2026.04 | 91.84 | 97.5 | |
| DivPrune w/ DSTPBackbone=Qwen3-VL-4B, Retention Ratio=33.3%2026.04 | 91.36 | 95.5 | |
| VisionZipBackbone=Qwen3-VL-4B, Retention Ratio=33.3%2026.04 | 90.41 | 96.5 | |
| PDrop+RerouteSource=Ours, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=66.7%2026.06 | 89.9 | — | |
| FastVBackbone=Qwen3-VL-4B, Retention Ratio=33.3%2026.04 | 87.98 | 94.3 | |
| VanillaSource=Qwen’25, Token Reduction=0%, Backbone=Qwen2.5-VL-7B2026.06 | 87.8 | — | |
| FastV+RerouteSource=Ours, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=66.7%2026.06 | 87.2 | — | |
| PDropSource=CVPR’25, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=66.7%2026.06 | 86.7 | — | |
| DivPruneBackbone=Qwen3-VL-4B, Retention Ratio=33.3%2026.04 | 86.12 | 91.2 | |
| PDropSource=CVPR’25, Token Reduction=66.7%, Backbone=Qwen2.5-VL-7B2026.06 | 85 | — | |
| PDrop+RerouteSource=Ours, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=77.8%2026.06 | 84.9 | — | |
| PDrop+RerouteSource=Ours, Token Reduction=66.7%, Backbone=Qwen2.5-VL-7B2026.06 | 84.8 | — | |
| FastVSource=ECCV’24, Token Reduction=66.7%, Backbone=Qwen2.5-VL-7B2026.06 | 82.8 | — | |
| FastVSource=ECCV’24, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=66.7%2026.06 | 82.8 | — | |
| FastV+RerouteSource=Ours, Token Reduction=66.7%, Backbone=Qwen2.5-VL-7B2026.06 | 82.5 | — | |
| PDropSource=CVPR’25, Token Reduction=77.8%, Backbone=Qwen2.5-VL-7B2026.06 | 82 | — | |
| FastV+RerouteSource=Ours, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=77.8%2026.06 | 82 | — | |
| PDrop+RerouteSource=Ours, Token Reduction=77.8%, Backbone=Qwen2.5-VL-7B2026.06 | 81.2 | — | |
| PDrop+RerouteSource=Ours, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=88.9%2026.06 | 80.8 | — | |
| PDropSource=CVPR’25, Token Reduction=88.9%, Backbone=Qwen2.5-VL-7B2026.06 | 79.4 | — | |
| PDropSource=CVPR’25, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=77.8%2026.06 | 79.3 | — | |
| FastV+RerouteSource=Ours, Token Reduction=77.8%, Backbone=Qwen2.5-VL-7B2026.06 | 78.8 | — | |
| FastV+RerouteSource=Ours, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=88.9%2026.06 | 78.5 | — | |
| FastVSource=ECCV’24, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=77.8%2026.06 | 78.3 | — | |
| PDrop+RerouteSource=Ours, Token Reduction=88.9%, Backbone=Qwen2.5-VL-7B2026.06 | 77.2 | — | |
| FastVSource=ECCV’24, Token Reduction=77.8%, Backbone=Qwen2.5-VL-7B2026.06 | 77.1 | — | |
| VISOR-TRFLOPs Savings=18×2026.03 | 75.4 | — | |
| VISORFLOPs Savings=8.6×2026.03 | 75.3 | — | |
| PDropSource=CVPR’25, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=88.9%2026.06 | 75.3 | — | |
| M³FLOPs Savings=8.0×2026.03 | 75.1 | — | |
| FastV+RerouteSource=Ours, Token Reduction=88.9%, Backbone=Qwen2.5-VL-7B2026.06 | 74.1 | — | |
| FastVSource=ECCV’24, Token Reduction=88.9%, Backbone=Qwen2.5-VL-7B2026.06 | 73.2 | — | |
| FastVSource=ECCV’24, Backbone=Qwen3.5-9B-Hybrid, Average Gated Attention Token Reduction=88.9%2026.06 | 73.1 | — | |
| BitDeltaα=1/16, Model Architecture=LLAVA-V1.5-13B2025.06 | 73 | — | |
| Alignedα=1, Model Architecture=LLAVA-V1.5-13B2025.06 | 72.8 | — | |
| Delta-CoMeα=1/16, Model Architecture=LLAVA-V1.5-13B2025.06 | 72.3 | — | |
| Q-VLMModel=LLaVA-13B, Bitwidth=W6A62024.10 | 72.27 | — | |
| PRINMIXα=1/16, Model Architecture=LLAVA-V1.5-13B2025.06 | 72.1 | — | |
| Full Precision BaselineModel=LLaVA-13B, Bitwidth=FP2024.10 | 71.61 | — | |
| QLoRAModel=LLaVA-13B, Bitwidth=W6A62024.10 | 71.52 | — | |
| Low-Rankα=1/16, Model Architecture=LLAVA-V1.5-13B2025.06 | 71.4 | — | |
| AWQModel=LLaVA-13B, Bitwidth=W6A62024.10 | 71.37 | — | |
| FEATHERVisual Token Budget=1922026.02 | 70.5 | 101.3 | |
| VanillaVisual Token Budget=5762026.02 | 69.6 | 100 | |
| VanillaBackbone=LLaVA-1.5-7B, Retained Tokens=5762026.06 | 69.5 | — | |
| HoloVBackbone=LLaVA-1.5-7B, Retained Tokens=642026.06 | 69.5 | — | |
| DCP-PruneBackbone=LLaVA-1.5-7B, Retained Tokens=162026.06 | 69.3 | — | |
| FlowCutBackbone=LLaVA-1.5-7B, Retained Tokens=642026.06 | 69.1 | — | |
| SwiftVLMVisual Token Budget=1922026.02 | 69 | 99.1 | |
| VisionZipBackbone=LLaVA-1.5-7B, Retained Tokens=642026.06 | 69 | — | |
| Q-VLMModel=LLaVA-13B, Bitwidth=W4A42024.10 | 68.84 | — | |
| VisionZipBackbone=LLaVA-1.5-7B, Retained Tokens=322026.06 | 68.8 | — | |
| HoloVBackbone=LLaVA-1.5-7B, Retained Tokens=322026.06 | 68.8 | — | |
| SCOPEBackbone=LLaVA-1.5-7B, Retained Tokens=642026.06 | 68.6 | — | |
| DCP-PruneBackbone=LLaVA-1.5-7B, Retained Tokens=322026.06 | 68.6 | — | |
| SwiftVLMVisual Token Budget=1282026.02 | 68.5 | 98.4 | |
| SCOPEBackbone=LLaVA-1.5-7B, Retained Tokens=322026.06 | 68.4 | — | |
| DCP-PruneBackbone=LLaVA-1.5-7B, Retained Tokens=642026.06 | 68.3 | — | |
| FlowCutBackbone=LLaVA-1.5-7B, Retained Tokens=322026.06 | 68.3 | — | |
| SCOPEBackbone=LLaVA-1.5-7B, Retained Tokens=162026.06 | 68.2 | — | |
| AWQModel=LLaVA-13B, Bitwidth=W4A42024.10 | 68.13 | — | |
| HoloVBackbone=LLaVA-1.5-7B, Retained Tokens=162026.06 | 68.1 | — | |
| QLoRAModel=LLaVA-13B, Bitwidth=W4A42024.10 | 68.04 | — | |
| PRUNESIDBackbone=LLaVA-1.5-7B, Retained Tokens=642026.06 | 67.8 | — | |
| PRUNESIDBackbone=LLaVA-1.5-7B, Retained Tokens=322026.06 | 67.8 | — | |
| VisionZipBackbone=LLaVA-1.5-7B, Retained Tokens=162026.06 | 67.8 | — | |
| DownsampleFLOPs Savings=4.0×2026.03 | 67.4 | — | |
| PRUNESIDBackbone=LLaVA-1.5-7B, Retained Tokens=162026.06 | 67.3 | — | |
| LLaVA-OVFLOPs Savings=1.0×2026.03 | 67.2 | — | |
| FlowCutBackbone=LLaVA-1.5-7B, Retained Tokens=162026.06 | 67.2 | — | |
| VisionZipFLOPs Savings=5.7×2026.03 | 67 | — | |
| Full Precision BaselineModel=LLaVA-7B, Bitwidth=FP2024.10 | 66.79 | — | |
| SparseVLMFLOPs Savings=4.5×2026.03 | 66.7 | — | |
| PyramidDropFLOPs Savings=4.2×2026.03 | 66.7 | — | |
| Q-VLMModel=LLaVA-7B, Bitwidth=W6A62024.10 | 66.67 | — | |
| HiREDFLOPs Savings=5.0×2026.03 | 66.3 | — | |
| QLoRAModel=LLaVA-7B, Bitwidth=W6A62024.10 | 66.16 | — | |
| AWQModel=LLaVA-7B, Bitwidth=W6A62024.10 | 65.87 | — | |
| VisionZip†FLOPs Savings=5.7×2026.03 | 65.8 | — | |
| VisPrunerFLOPs Savings=5.7×2026.03 | 65.7 | — | |
| Q-VLMModel=LLaVA-7B, Bitwidth=W4A42024.10 | 57.7 | — | |
| AWQModel=LLaVA-7B, Bitwidth=W4A42024.10 | 56.73 | — | |
| QLoRAModel=LLaVA-7B, Bitwidth=W4A42024.10 | 56.5 | — |