Scientific Question Answering on ScienceQA
94.4AccuracyVanilla
Evaluation results
| Method | Links | |
|---|---|---|
| VanillaBackbone=Qwen3-VL-8B, TFLOPs=100%2026.06 | 94.4 | |
| Op-SkipBackbone=Qwen3-VL-8B, TFLOPs=66%, N=202026.06 | 94.35 | |
| ShortVBackbone=Qwen3-VL-8B, TFLOPs=58%, N=202026.06 | 92.61 | |
| APETBackbone=Qwen3-VL-8B, TFLOPs=66%2026.06 | 92.56 | |
| VSkipBackbone=Qwen3-VL-8B, TFLOPs=76%, N=202026.06 | 92.46 | |
| V2DropBackbone=Qwen3-VL-8B, TFLOPs=66%2026.06 | 91.72 | |
| MASQuantModel=Qwen2.5-VL-7B, Bits=W8A82026.03 | 88.6 | |
| MBQModel=Qwen2.5-VL-7B, Bits=W8A82026.03 | 88.5 | |
| DenseModel=Qwen2.5-VL-7B, Bits=FP162026.03 | 88.4 | |
| SQModel=Qwen2.5-VL-7B, Bits=W8A82026.03 | 88.2 | |
| RTNModel=Qwen2.5-VL-7B, Bits=W8A82026.03 | 88.1 | |
| AWQModel=Qwen2.5-VL-7B, Bits=W4A162026.03 | 87.8 | |
| MBQModel=Qwen2.5-VL-7B, Bits=W4A162026.03 | 87.8 | |
| MASQuantModel=Qwen2.5-VL-7B, Bits=W4A162026.03 | 87.8 | |
| VanillaBackbone=Qwen2.5-VL-7B, TFLOPs=100%2026.06 | 87.51 | |
| APETBackbone=Qwen2.5-VL-7B, TFLOPs=55%2026.06 | 87.51 | |
| V2DropBackbone=Qwen2.5-VL-7B, TFLOPs=55%2026.06 | 87.06 | |
| MBQModel=Qwen2.5-VL-7B, Bits=W4A82026.03 | 86 | |
| VSkipBackbone=Qwen2.5-VL-7B, TFLOPs=67%, N=162026.06 | 85.72 | |
| MASQuantModel=Qwen2.5-VL-7B, Bits=W4A82026.03 | 85.7 | |
| Op-SkipBackbone=Qwen2.5-VL-7B, TFLOPs=55%, N=202026.06 | 85.52 | |
| RTNModel=Qwen2.5-VL-7B, Bits=W4A82026.03 | 85.2 | |
| SQModel=Qwen2.5-VL-7B, Bits=W4A82026.03 | 83.3 | |
| MASQuantModel=Qwen2.5-VL-3B, Bits=W4A162026.03 | 82.4 | |
| RTNModel=Qwen2.5-VL-3B, Bits=W8A82026.03 | 82.4 | |
| MASQuantModel=Qwen2.5-VL-3B, Bits=W8A82026.03 | 82.4 | |
| SQModel=Qwen2.5-VL-3B, Bits=W8A82026.03 | 82.2 | |
| DenseModel=Qwen2.5-VL-3B, Bits=FP162026.03 | 81.9 | |
| MBQModel=Qwen2.5-VL-3B, Bits=W8A82026.03 | 81.8 | |
| AWQModel=Qwen2.5-VL-3B, Bits=W4A162026.03 | 81.7 | |
| MBQModel=Qwen2.5-VL-3B, Bits=W4A162026.03 | 81.5 | |
| RTNModel=Qwen2.5-VL-3B, Bits=W4A162026.03 | 81.3 | |
| RTNModel=Qwen2.5-VL-7B, Bits=W4A162026.03 | 81.3 | |
| VTWBackbone=Qwen3-VL-8B, TFLOPs=58%, K=162026.06 | 81.11 | |
| VTWBackbone=Qwen2.5-VL-7B, TFLOPs=50%, K=162026.06 | 80.32 | |
| MBQModel=Qwen2.5-VL-7B, Bits=W4A62026.03 | 80.1 | |
| MASQuantModel=Qwen2.5-VL-7B, Bits=W4A62026.03 | 79.7 | |
| SQModel=Qwen2.5-VL-7B, Bits=W4A62026.03 | 78.6 | |
| ShortVBackbone=Qwen2.5-VL-7B, TFLOPs=50%, N=202026.06 | 78.38 | |
| MASQuantModel=Qwen2.5-VL-3B, Bits=W4A82026.03 | 77.9 | |
| MBQModel=Qwen2.5-VL-3B, Bits=W4A82026.03 | 76.7 | |
| LLaVA-v1.6 (7B) w/ STICBase Model=LLaVA-v1.6 (7B), Strategy=STIC2024.05 | 75.3 | |
| SQModel=Qwen2.5-VL-3B, Bits=W4A82026.03 | 72.1 | |
| MASQuantModel=Qwen2.5-VL-3B, Bits=W4A62026.03 | 71.2 | |
| PDropToken Retention Rate=64, Base Model=LLaVA-1.5-7B2025.06 | 69.96 | |
| SQModel=Qwen2.5-VL-3B, Bits=W4A62026.03 | 69.9 | |
| VTWBackbone=LLaVA-1.5-7B, TFLOPs=55%, K=162026.06 | 69.71 | |
| SparseVLM-v1.5Token Retention Rate=64, Base Model=LLaVA-1.5-7B2025.06 | 69.61 | |
| PDropToken Retention Rate=192, Base Model=LLaVA-1.5-7B2025.06 | 69.56 | |
| LLaVA-1.5-7BToken Retention Rate=576, Base Model=LLaVA-1.5-7B2025.06 | 69.51 | |
| LLaVA-v1.5 (7B) w/ STICBase Model=LLaVA-v1.5 (7B), Strategy=STIC2024.05 | 69.5 | |
| VanillaBackbone=LLaVA-1.5-7B, TFLOPs=100%2026.06 | 69.46 | |
| VisionDropToken Retention Rate=32, Base Model=LLaVA-1.5-7B2025.06 | 69.41 | |
| PDropToken Retention Rate=128, Base Model=LLaVA-1.5-7B2025.06 | 69.36 | |
| FastVToken Retention Rate=64, Base Model=LLaVA-1.5-7B2025.06 | 69.36 | |
| VisionDropToken Retention Rate=64, Base Model=LLaVA-1.5-7B2025.06 | 69.31 | |
| VisPrunerToken Retention Rate=32, Base Model=LLaVA-1.5-7B2025.06 | 69.26 | |
| VSkip+Backbone=LLaVA-1.5-7B, TFLOPs=76%, N=202026.06 | 69.26 | |
| VSkipBackbone=LLaVA-1.5-7B, TFLOPs=76%, N=202026.06 | 69.16 | |
| Op-SkipBackbone=LLaVA-1.5-7B, TFLOPs=66%, N=202026.06 | 69.16 | |
| V2DropBackbone=LLaVA-1.5-7B, TFLOPs=66%2026.06 | 69.12 | |
| VisionDropToken Retention Rate=192, Base Model=LLaVA-1.5-7B2025.06 | 69.06 | |
| VisPrunerToken Retention Rate=128, Base Model=LLaVA-1.5-7B2025.06 | 69.01 | |
| VisionZipToken Retention Rate=64, Base Model=LLaVA-1.5-7B2025.06 | 69.01 | |
| APETBackbone=LLaVA-1.5-7B, TFLOPs=66%2026.06 | 68.93 | |
| LLaVA-v1.6 (7B)Model=LLaVA-v1.6 (7B)2024.05 | 68.9 | |
| SparseVLM-v1.5Token Retention Rate=192, Base Model=LLaVA-1.5-7B2025.06 | 68.86 | |
| VisionZipToken Retention Rate=192, Base Model=LLaVA-1.5-7B2025.06 | 68.86 | |
| VisionZipToken Retention Rate=128, Base Model=LLaVA-1.5-7B2025.06 | 68.82 | |
| VisPrunerToken Retention Rate=64, Base Model=LLaVA-1.5-7B2025.06 | 68.82 | |
| LLaVA-v1.5 (7B) w/ POVIDBase Model=LLaVA-v1.5 (7B), Strategy=POVID2024.05 | 68.8 | |
| VisionZipToken Retention Rate=32, Base Model=LLaVA-1.5-7B2025.06 | 68.72 | |
| SparseVLM-v1.5Token Retention Rate=128, Base Model=LLaVA-1.5-7B2025.06 | 68.57 | |
| VisPrunerToken Retention Rate=192, Base Model=LLaVA-1.5-7B2025.06 | 68.52 | |
| VisionDropToken Retention Rate=128, Base Model=LLaVA-1.5-7B2025.06 | 68.52 | |
| FastVToken Retention Rate=128, Base Model=LLaVA-1.5-7B2025.06 | 68.37 | |
| ShortVBackbone=LLaVA-1.5-7B, TFLOPs=55%, N=202026.06 | 68.27 | |
| FastVToken Retention Rate=192, Base Model=LLaVA-1.5-7B2025.06 | 68.07 | |
| LLaVA-v1.5 (7B)Model=LLaVA-v1.5 (7B)2024.05 | 66.8 | |
| MBQModel=Qwen2.5-VL-3B, Bits=W4A62026.03 | 64.7 | |
| mPLUG-OWL2 (7B)Model=mPLUG-OWL2 (7B)2024.05 | 64.5 | |
| InstructBLIP (7B)Model=InstructBLIP (7B)2024.05 | 60.5 | |
| RTNModel=Qwen2.5-VL-3B, Bits=W4A82026.03 | 0 |