Visual Question Answering on VQA v2 (test val)
78.5AccuracyNative
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| NativeRetained Tokens=5762026.03 | 78.5 | 1 | 1 | 1 | 76.5 | 100 | — | — | — | — | — | — | — | |
| EvoPruneRetained Tokens=1282026.03 | 76.8 | 1.1 | 0.77 | 0.84 | 74.9 | 97.9 | — | — | — | — | — | — | — | |
| CDPrunerRetained Tokens=1282026.03 | 76.6 | 1.41 | 0.77 | 0.91 | 73.9 | 96.6 | — | — | — | — | — | — | — | |
| Firebolt-VLLLM=LFM2, Parameters=0.8B2026.04 | 76.6 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MoE-LLaVALLM=Qwen, Parameters=2.2B2026.04 | 76.2 | — | — | — | — | — | — | — | — | — | — | — | — | |
| DivPruneRetained Tokens=1282026.03 | 76 | 1.16 | 0.77 | 0.85 | 72.9 | 95.3 | — | — | — | — | — | — | — | |
| VisPrunerRetained Tokens=1282026.03 | 75.8 | 1.32 | 0.77 | 0.9 | 73.2 | 95.7 | — | — | — | — | — | — | — | |
| EvoPruneRetained Tokens=642026.03 | 75.6 | 1.07 | 0.73 | 0.77 | 72.9 | 95.3 | — | — | — | — | — | — | — | |
| FasterVLMRetained Tokens=1282026.03 | 75.5 | 1.15 | 0.77 | 0.85 | 72.3 | 94.6 | — | — | — | — | — | — | — | |
| CDPrunerRetained Tokens=642026.03 | 75.4 | 1.38 | 0.73 | 0.86 | 72.2 | 94.4 | — | — | — | — | — | — | — | |
| DARTRetained Tokens=1282026.03 | 74.7 | 1.2 | 0.77 | 0.88 | 70.9 | 92.7 | — | — | — | — | — | — | — | |
| DivPruneRetained Tokens=642026.03 | 74.1 | 1.12 | 0.73 | 0.79 | 70.4 | 92 | — | — | — | — | — | — | — | |
| VisPrunerRetained Tokens=642026.03 | 73.9 | 1.35 | 0.73 | 0.86 | 69.9 | 91.3 | — | — | — | — | — | — | — | |
| Mean NetworkBackbone=BEiT-3-base, Rel. Time=1.0×2026.06 | 73.8 | — | — | — | — | — | 7.67 | 13.8 | 22.1 | 2.72 | — | 3.83 | 0.4 | |
| + Temp. Sc.Backbone=BEiT-3-base, Rel. Time=1.0×2026.06 | 73.8 | — | — | — | — | — | 7.67 | 13.8 | 22.1 | 2.72 | — | 3.74 | 0.4 | |
| StreamliningBackbone=BEiT-3-base, Rel. Time=2.0×2026.06 | 73.8 | — | — | — | — | — | 7.67 | 13.8 | 22.1 | 2.72 | — | 3.79 | 0.4 | |
| CVPBackbone=BEiT-3-base, Rel. Time=2.3×2026.06 | 73.8 | — | — | — | — | — | 7.57 | 17.3 | 24.6 | 2.7 | — | 3.17 | 0.4 | |
| CDPrunerRetained Tokens=322026.03 | 73.6 | 1.22 | 0.71 | 0.8 | 69.6 | 91 | — | — | — | — | — | — | — | |
| 4 samplesBackbone=BEiT-3-base, Rel. Time=4.0×2026.06 | 73.5 | — | — | — | — | — | 7.76 | 15.6 | 22.7 | 2.71 | — | 2.15 | 0.4 | |
| 2 samplesBackbone=BEiT-3-base, Rel. Time=2.0×2026.06 | 73.3 | — | — | — | — | — | 7.94 | 13.7 | 21.8 | 2.74 | — | 2.53 | 0.4 | |
| EvoPruneRetained Tokens=322026.03 | 73.1 | 1.06 | 0.71 | 0.75 | 68.7 | 89.8 | — | — | — | — | — | — | — | |
| FasterVLMRetained Tokens=642026.03 | 72.7 | 1.09 | 0.73 | 0.79 | 68.2 | 89.2 | — | — | — | — | — | — | — | |
| VisPrunerRetained Tokens=322026.03 | 72.1 | 1.35 | 0.71 | 0.84 | 65.2 | 85.2 | — | — | — | — | — | — | — | |
| LLaVA-PhiLLM=Phi 2, Parameters=2.7B2026.04 | 71.4 | — | — | — | — | — | — | — | — | — | — | — | — | |
| DARTRetained Tokens=642026.03 | 71.3 | 1.13 | 0.73 | 0.8 | 66.2 | 86.6 | — | — | — | — | — | — | — | |
| DivPruneRetained Tokens=322026.03 | 71.2 | 1.1 | 0.71 | 0.77 | 66.8 | 87.4 | — | — | — | — | — | — | — | |
| FasterVLMRetained Tokens=322026.03 | 70.9 | 1.09 | 0.71 | 0.77 | 64.5 | 84.4 | — | — | — | — | — | — | — | |
| CVPBackbone=ViLT, Rel. Time=2.2×2026.06 | 69.3 | — | — | — | — | — | 10.3 | 10.8 | 17.4 | 3.02 | — | 3.69 | 0.46 | |
| Mean NetworkBackbone=ViLT, Rel. Time=1.0×2026.06 | 69.2 | — | — | — | — | — | 10.53 | 2.4 | 13.3 | 3.1 | — | 7.72 | 0.48 | |
| + Temp. Sc.Backbone=ViLT, Rel. Time=1.0×2026.06 | 69.2 | — | — | — | — | — | 10.53 | 2.5 | 13.3 | 3.1 | — | 7.68 | 0.48 | |
| StreamliningBackbone=ViLT, Rel. Time=2.0×2026.06 | 69.2 | — | — | — | — | — | 10.53 | 2.6 | 13.3 | 3.1 | — | 7.7 | 0.48 | |
| 4 samplesBackbone=ViLT, Rel. Time=4.0×2026.06 | 68.6 | — | — | — | — | — | 10.87 | 10.5 | 15.7 | 3.07 | — | 2.06 | 0.47 | |
| 2 samplesBackbone=ViLT, Rel. Time=2.0×2026.06 | 68 | — | — | — | — | — | 11.43 | 7.6 | 13.2 | 3.13 | — | 2.9 | 0.48 | |
| DARTRetained Tokens=322026.03 | 67.1 | 1.09 | 0.71 | 0.76 | 59.8 | 78.1 | — | — | — | — | — | — | — | |
| OpenFlamingo v2LLM=MPT, Parameters=9.0B2026.04 | 60.4 | — | — | — | — | — | — | — | — | — | — | — | — | |
| IDEFICSLLM=LLaMA, Parameters=9.0B2026.04 | 60 | — | — | — | — | — | — | — | — | — | — | — | — | |
| FastVLLM=Vicuna, Parameters=7.0B2026.04 | 55 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Kosmos-2LLM=GPT 2, Parameters=1.7B2026.04 | 45.6 | — | — | — | — | — | — | — | — | — | — | — | — |