Visual Question Answering on ScienceQA
98.18AccuracyCont-Squeeze (128 -> 1)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Cont-Squeeze (128 -> 1)Optimization steps=+700 steps2026.02 | 98.18 | — | — | |
| In-Squeeze (128 ... -> 1)Optimization mode=Min steps2026.02 | 98.06 | — | — | |
| Direct Fine-tuning (r=1)Optimization steps=+0 steps2026.02 | 98 | — | — | |
| Direct Fine-tuning (r=1)Optimization steps=+200 steps2026.02 | 98 | — | — | |
| Direct Fine-tuning (r=1)Optimization steps=+700 steps2026.02 | 97.94 | — | — | |
| Cont-Squeeze (128 -> 1)Optimization steps=+200 steps2026.02 | 97.58 | — | — | |
| In-Squeeze (128 ... -> 1)Optimization mode=Standard2026.02 | 97.55 | — | — | |
| FP16 BaselineModel=InternVL2-8B, Bitwidth=FP162026.03 | 97.07 | — | — | |
| QIGModel=InternVL2-8B, Bitwidth=W4A82026.03 | 96.73 | — | — | |
| VanillaModel=InternVL3.5-8B, Token Retention Ratio=100%2026.04 | 96.7 | — | — | |
| MBQModel=InternVL2-8B, Bitwidth=W4A82026.03 | 96.38 | — | — | |
| RTNModel=InternVL2-8B, Bitwidth=W3A162026.03 | 96.28 | — | — | |
| RTNModel=InternVL2-8B, Bitwidth=W4A82026.03 | 96.22 | — | — | |
| Cont-Squeeze (128 -> 1)Optimization steps=+0 steps2026.02 | 96.17 | — | — | |
| QIGModel=InternVL2-8B, Bitwidth=W3A162026.03 | 96.03 | — | — | |
| MBQModel=InternVL2-8B, Bitwidth=W3A162026.03 | 95.93 | — | — | |
| FP16 BaselineModel=LLaVA-onevision-7B, Bitwidth=FP162026.03 | 95.88 | — | — | |
| AWQModel=InternVL2-8B, Bitwidth=W3A162026.03 | 95.88 | — | — | |
| SQModel=InternVL2-8B, Bitwidth=W4A82026.03 | 95.88 | — | — | |
| QIGModel=LLaVA-onevision-7B, Bitwidth=W3A162026.03 | 95.29 | — | — | |
| MBQModel=LLaVA-onevision-7B, Bitwidth=W3A162026.03 | 94.89 | — | — | |
| MBQModel=LLaVA-onevision-7B, Bitwidth=W4A82026.03 | 94.7 | — | — | |
| RTNModel=LLaVA-onevision-7B, Bitwidth=W3A162026.03 | 94.55 | — | — | |
| VisionLLM v2-chatBackbone=Vicuna-7B, #Data=22M2024.06 | 94.4 | — | — | |
| GPTQModel=InternVL2-8B, Bitwidth=W3A162026.03 | 94.3 | — | — | |
| QIGModel=LLaVA-onevision-7B, Bitwidth=W4A82026.03 | 94.25 | — | — | |
| RTNModel=LLaVA-onevision-7B, Bitwidth=W4A82026.03 | 94.1 | — | — | |
| SQModel=LLaVA-onevision-7B, Bitwidth=W4A82026.03 | 93.51 | — | — | |
| VanillaModel=Qwen3-VL-4B, Token Retention Ratio=100%2026.04 | 93.42 | — | — | |
| Gemma 3 4B ITZero-shot=true2026.02 | 92.3 | — | — | |
| GPTQModel=LLaVA-onevision-7B, Bitwidth=W3A162026.03 | 92.12 | — | — | |
| VisionZip w/ DSTPModel=Qwen3-VL-4B, Token Retention Ratio=33.3%2026.04 | 92.08 | — | 98.6 | |
| FastV w/ DSTPModel=Qwen3-VL-4B, Token Retention Ratio=33.3%2026.04 | 91.84 | — | 98.3 | |
| DivPrune w/ DSTPModel=Qwen3-VL-4B, Token Retention Ratio=33.3%2026.04 | 91.36 | — | 97.8 | |
| FastV w/ DSTPModel=Qwen3-VL-4B, Token Retention Ratio=22.2%2026.04 | 91.27 | — | 97.7 | |
| CogVLM-13B-ChatBackbone=Vicuna-7B, #Data=>1.5B2024.06 | 91.2 | — | — | |
| GMAILBase Model=LLaMA-32026.02 | 91.2 | — | — | |
| DivPrune w/ DSTPModel=Qwen3-VL-4B, Token Retention Ratio=22.2%2026.04 | 90.79 | — | 97.2 | |
| DivPrune w/ DSTPModel=InternVL3.5-8B, Token Retention Ratio=33.3%2026.04 | 90.75 | — | 93.8 | |
| VisionZipModel=Qwen3-VL-4B, Token Retention Ratio=33.3%2026.04 | 90.41 | — | 96.8 | |
| VisionZip w/ DSTPModel=Qwen3-VL-4B, Token Retention Ratio=22.2%2026.04 | 90.27 | — | 96.6 | |
| VisionZip w/ DSTPModel=InternVL3.5-8B, Token Retention Ratio=33.3%2026.04 | 90.19 | — | 93.3 | |
| DivPruneModel=InternVL3.5-8B, Token Retention Ratio=33.3%2026.04 | 89.98 | — | 93.1 | |
| VisionZipModel=InternVL3.5-8B, Token Retention Ratio=33.3%2026.04 | 88.79 | — | 91.8 | |
| VisionZipModel=Qwen3-VL-4B, Token Retention Ratio=22.2%2026.04 | 88.65 | — | 94.9 | |
| LLaMA-32026.02 | 88.5 | — | — | |
| VisionZip w/ DSTPModel=InternVL3.5-8B, Token Retention Ratio=22.2%2026.04 | 88.07 | — | 91.1 | |
| Visual-SR1Backbone=Qwen2.5-VL-7B2025.09 | 88 | — | — | |
| FastVModel=Qwen3-VL-4B, Token Retention Ratio=33.3%2026.04 | 87.98 | — | 94.2 | |
| FastV w/ DSTPModel=InternVL3.5-8B, Token Retention Ratio=33.3%2026.04 | 87.65 | — | 90.6 | |
| GMAILBase Model=LLaVA2026.02 | 87.6 | — | — | |
| MaD-MixModel size=7B, Evaluation protocol=0-shot2026.02 | 87.26 | — | — | |
| DivPrune w/ DSTPModel=InternVL3.5-8B, Token Retention Ratio=22.2%2026.04 | 87.19 | — | 90.2 | |
| FinetunedBackbone=LLaVA, Adapter Rank=162026.03 | 86.6 | — | — | |
| DivPruneModel=InternVL3.5-8B, Token Retention Ratio=22.2%2026.04 | 86.5 | — | 89.5 | |
| DivPruneModel=Qwen3-VL-4B, Token Retention Ratio=33.3%2026.04 | 86.12 | — | 92.2 | |
| AvgModel size=7B, Evaluation protocol=0-shot2026.02 | 86.07 | — | — | |
| OtterHD-8BMaxRes.=1024x1024, AR.=Any, TFLOPS=21.32024.03 | 86 | — | — | |
| FastV w/ DSTPModel=InternVL3.5-8B, Token Retention Ratio=22.2%2026.04 | 85.74 | — | 88.7 | |
| SeProD2026.05 | 85.4 | — | — | |
| LLaVA2026.02 | 85.2 | — | — | |
| H-GIVRModel=Qwen2.5vl:7b, Prompt=Ours2026.02 | 85.12 | — | — | |
| FP16 BaselineModel=Qwen2-VL-7B, Bitwidth=FP162026.03 | 85.03 | — | — | |
| AVLM-2BModel Scale=2B2026.06 | 85.03 | — | — | |
| VisionZipModel=InternVL3.5-8B, Token Retention Ratio=22.2%2026.04 | 84.54 | — | 87.4 | |
| Mini-o32026.05 | 84.5 | — | — | |
| Qwen3-VL-2B-Instruct + MoDA (ours)LLM=Qwen3-2B2025.06 | 84.2 | — | — | |
| FastVModel=Qwen3-VL-4B, Token Retention Ratio=22.2%2026.04 | 83.69 | — | 89.6 | |
| DeFactoBackbone=Qwen2.5-VL-7B2025.09 | 83.6 | — | — | |
| UniformModel size=7B, Evaluation protocol=0-shot2026.02 | 83.44 | — | — | |
| FusedModel size=7B, Evaluation protocol=0-shot2026.02 | 83.29 | — | — | |
| DivPruneModel=Qwen3-VL-4B, Token Retention Ratio=22.2%2026.04 | 83.02 | — | 88.9 | |
| CL-MOEBackbone=LLaVA-1.5-7B2026.04 | 83 | — | — | |
| Qwen2.5-VLBackbone=Qwen2.5-VL-7B2025.09 | 83 | — | — | |
| AWQModel=LLaVA-onevision-7B, Bitwidth=W3A162026.03 | 82.2 | — | — | |
| GPT-4V2024.06 | 82.1 | — | — | |
| ViCropBackbone=LLaVA-1.5 (Vicuna-7B)2025.09 | 82.1 | — | — | |
| LLaVA-MoRE SigLIP-S2 + MoDA (ours)LLM=LLaMA 3.1-8B2025.06 | 81.9 | — | — | |
| MBQModel=Qwen2-VL-7B, Bitwidth=W3A162026.03 | 81.85 | — | — | |
| GPTQModel=Qwen2-VL-7B, Bitwidth=W3A162026.03 | 81.76 | — | — | |
| Qwen2.5-Omni-3BModel Scale=3B2026.06 | 81.66 | — | — | |
| QIGModel=Qwen2-VL-7B, Bitwidth=W3A162026.03 | 81.61 | — | — | |
| FastVModel=InternVL3.5-8B, Token Retention Ratio=33.3%2026.04 | 81.6 | — | 84.4 | |
| GeminiProVision2024.06 | 81.4 | — | — | |
| AIM-CoTBackbone=Qwen2.5-VL-32B, Shot=1-shot2025.09 | 81.3 | — | — | |
| RTNModel=Qwen2-VL-7B, Bitwidth=W3A162026.03 | 81.06 | — | — | |
| AWQModel=Qwen2-VL-7B, Bitwidth=W3A162026.03 | 81.01 | — | — | |
| Metis-HOMEBackbone=LLaVA-1.5-7B2026.04 | 80.5 | — | — | |
| MoMEBackbone=LLaVA-1.5-7B2026.04 | 80.4 | — | — | |
| FastVModel=InternVL3.5-8B, Token Retention Ratio=22.2%2026.04 | 80.35 | — | 83.1 | |
| QIGModel=Qwen2-VL-7B, Bitwidth=W4A82026.03 | 80.17 | — | — | |
| RTNModel=Qwen2-VL-7B, Bitwidth=W4A82026.03 | 79.62 | — | — | |
| Dragonfly (Ours)Backbone=Llama3-8B, #Data=2.9M2024.06 | 79.5 | — | — | |
| DragonflyBackbone=Llama3-8B, #Data=2.9M2024.06 | 79.5 | — | — | |
| Qwen3-VL-2B-InstructLLM=Qwen3-2B2025.06 | 79.3 | — | — | |
| ICoTBackbone=Qwen2.5-VL-32B, Shot=1-shot2025.09 | 79.2 | — | — | |
| MoVALLM=Hermes-Yi-34B, Params=38B, Model Type=Specialist, Zero-shot=true2024.04 | 79 | — | — | |
| SQModel=Qwen2-VL-7B, Bitwidth=W4A82026.03 | 78.98 | — | — | |
| MBQModel=Qwen2-VL-7B, Bitwidth=W4A82026.03 | 78.93 | — | — | |
| H-GIVRModel=Llama3.2-vision:11b, Prompt=Ours2026.02 | 78.9 | — | — |