Visual Question Answering on ScienceQA img
86.2AccuracyUnified-IO 2
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Unified-IO 2LLM=UIO-2-6.8B, Res.=3842024.03 | 86.2 | — | |
| LASER-noQModel Architecture=SmolVLM 2B, Parameter Ratio (rho1)=90%, Precision=FP162026.05 | 84.58 | 74.51 | |
| FP16Model Architecture=SmolVLM 2B, Parameter Ratio (rho1)=100%, Precision=FP162026.05 | 84.58 | 76.53 | |
| FP16Model=SmolVLM 2B, Ratio=100%2026.05 | 84.53 | — | |
| LASER-noQModel Architecture=SmolVLM 2B, Parameter Ratio (rho1)=80%, Precision=FP162026.05 | 83.69 | 74.51 | |
| IO-SVDModel=SmolVLM 2B, Ratio=90%2026.05 | 83.34 | — | |
| IO-SVDModel=SmolVLM 2B, Ratio=80%2026.05 | 82.65 | — | |
| IO-SVDModel=SmolVLM 2B, Ratio=70%2026.05 | 82.3 | — | |
| LASER-noQModel Architecture=SmolVLM 2B, Parameter Ratio (rho1)=70%, Precision=FP162026.05 | 82 | 74.51 | |
| LASER-noQModel Architecture=SmolVLM 2B, Parameter Ratio (rho1)=60%, Precision=FP162026.05 | 81.11 | 74.51 | |
| NuwaBackbone=QWEN-2.5-VL 7B, Average Tokens=50%2026.02 | 78.82 | — | |
| Unified-IO 2Source=CVPR’242024.07 | 78.6 | — | |
| NuwaBackbone=QWEN-2.5-VL 7B, Average Tokens=25%2026.02 | 78.58 | — | |
| LASER-noQModel Architecture=SmolVLM 2B, Parameter Ratio (rho1)=50%, Precision=FP162026.05 | 78.14 | 74.51 | |
| NuwaBackbone=QWEN-2.5-VL 7B, Average Tokens=75%2026.02 | 77.98 | — | |
| VanillaBackbone=QWEN-2.5-VL 7B, Average Tokens=100%2026.02 | 77.8 | — | |
| INF-LLaVA*Source=Ours, larger_dataset=true2024.07 | 77.44 | — | |
| QSVD-noQModel Architecture=SmolVLM 2B, Parameter Ratio (rho1)=90%, Precision=FP162026.05 | 77 | 34.98 | |
| WSVD-noQModel Architecture=SmolVLM 2B, Parameter Ratio (rho1)=90%, Precision=FP162026.05 | 76.3 | 51.58 | |
| VT-LMMLLM=Mistral-7B, Res.=3362024.03 | 75.9 | — | |
| INF-LLaVASource=Ours2024.07 | 75.71 | — | |
| LASER-noQModel Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=50%, Precision=FP162026.05 | 73.67 | 72.31 | |
| WSVD-noQModel Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=70%, Precision=FP162026.05 | 73.57 | 72.17 | |
| WSVD-noQModel Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=60%, Precision=FP162026.05 | 73.48 | 72.17 | |
| LASER-noQModel Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=70%, Precision=FP162026.05 | 73.38 | 72.31 | |
| WSVD-noQModel Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=50%, Precision=FP162026.05 | 73.28 | 72.17 | |
| FP16Model Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=100%, Precision=FP162026.05 | 73.23 | 72.27 | |
| LASER-noQModel Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=90%, Precision=FP162026.05 | 73.13 | 72.31 | |
| LASER-noQModel Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=60%, Precision=FP162026.05 | 73.13 | 72.31 | |
| LASER-noQModel Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=80%, Precision=FP162026.05 | 73.03 | 72.31 | |
| WSVD-noQModel Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=80%, Precision=FP162026.05 | 72.98 | 72.17 | |
| WSVD-noQModel Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=90%, Precision=FP162026.05 | 72.88 | 72.17 | |
| IO-SVDModel=LLaVA1.5 13B, Ratio=50%2026.05 | 72.68 | — | |
| SVD-LLMModel Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=90%, Precision=FP162026.05 | 72.53 | 70.94 | |
| IO-SVDModel=LLaVA1.5 13B, Ratio=70%2026.05 | 72.29 | — | |
| SVD-LLMModel Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=80%, Precision=FP162026.05 | 72.24 | 70.94 | |
| IO-SVDModel=LLaVA1.5 13B, Ratio=80%2026.05 | 72.14 | — | |
| IO-SVDModel=LLaVA1.5 13B, Ratio=60%2026.05 | 72.14 | — | |
| QSVD-noQModel Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=80%, Precision=FP162026.05 | 72.14 | 71.44 | |
| QSVD-noQModel Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=60%, Precision=FP162026.05 | 72.14 | 71.44 | |
| QSVD-noQModel Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=90%, Precision=FP162026.05 | 71.94 | 71.44 | |
| FP16Model=LLaVA1.5 13B, Ratio=100%2026.05 | 71.83 | — | |
| IO-SVDModel=LLaVA1.5 13B, Ratio=90%2026.05 | 71.79 | — | |
| QSVD-noQModel Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=50%, Precision=FP162026.05 | 71.79 | 71.44 | |
| WSVD-noQModel Architecture=SmolVLM 2B, Parameter Ratio (rho1)=80%, Precision=FP162026.05 | 71.74 | 51.58 | |
| SVD-LLMModel Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=70%, Precision=FP162026.05 | 71.74 | 70.94 | |
| QSVD-noQModel Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=70%, Precision=FP162026.05 | 71.74 | 71.44 | |
| LLaVA-v1.5LLM=Vicuna-13B, Res.=3362024.03 | 71.6 | — | |
| LLaVA-1.5LLM=Vicuna-13B, Activated Parameters=13B, Resolution=3362024.01 | 71.6 | — | |
| SVD-LLMModel Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=60%, Precision=FP162026.05 | 71.15 | 70.94 | |
| AnyMAL-70BSource=arXiv’232024.07 | 70.8 | — | |
| SVD-LLMModel Architecture=LLaVA-Next 13B, Parameter Ratio (rho1)=50%, Precision=FP162026.05 | 70.55 | 70.94 | |
| MoE-LLaVA-2.7Bx4-Top2†LLM=Phi-2-2.7B, Activated Parameters=3.6B, Resolution=3842024.01 | 70.3 | — | |
| MonkeySource=CVPR’242024.07 | 69.4 | — | |
| AcFormerSource=arXiv’242024.07 | 69.4 | — | |
| SugarLLM=Vicuna-7B, Resolution=3362024.11 | 69.4 | — | |
| TIVESource=arXiv’242024.07 | 69.2 | — | |
| mPLUG-Owl2Source=CVPR’242024.07 | 68.7 | — | |
| MoE-LLaVA-2.7Bx4-Top2LLM=Phi-2-2.7B, Res.=3362024.03 | 68.5 | — | |
| MoE-LLaVA-2.7Bx4-Top2LLM=Phi-2-2.7B, Activated Parameters=3.6B, Resolution=3362024.01 | 68.5 | — | |
| LoRA-SparseSource=CVPR’242024.07 | 68.4 | — | |
| LLaVA-PhiSource=arXiv’242024.07 | 68.4 | — | |
| LLaVA-PhiLLM=Phi-2-2.7B, Activated Parameters=2.7B, Resolution=3362024.01 | 68.4 | — | |
| QSVD-noQModel Architecture=LLaVA-v1.5 7B, Parameter Ratio (rho1)=80%, Precision=FP162026.05 | 68.22 | 62.64 | |
| Qwen-VL-ChatSource=arXiv’232024.07 | 68.2 | — | |
| Qwen-VL-ChatLLM=Qwen-7B, Res.=4482024.03 | 68.2 | — | |
| Qwen-VL-ChatLLM=Qwen-7B, Resolution=4482024.11 | 68.2 | — | |
| VILA-7BLLM=Llama-2-7B, Resolution=3362024.11 | 68.2 | — | |
| WSVDModel=LLaVA1.5 7B, Ratio=90%2026.05 | 68.17 | — | |
| IO-SVDModel=LLaVA1.5 7B, Ratio=70%2026.05 | 68.17 | — | |
| WSVD-noQModel Architecture=LLaVA-v1.5 7B, Parameter Ratio (rho1)=90%, Precision=FP162026.05 | 68.17 | 63.54 | |
| LASER-noQModel Architecture=LLaVA-v1.5 7B, Parameter Ratio (rho1)=90%, Precision=FP162026.05 | 68.12 | 63.81 | |
| VT-LMMLLM=Vicuna-7B, Res.=3362024.03 | 68.1 | — | |
| IO-SVDModel=LLaVA1.5 7B, Ratio=80%2026.05 | 68.07 | — | |
| LASER-noQModel Architecture=LLaVA-v1.5 7B, Parameter Ratio (rho1)=80%, Precision=FP162026.05 | 68.02 | 63.81 | |
| FP16Model=LLaVA1.5 7B, Ratio=100%2026.05 | 68.01 | — | |
| FP16Model Architecture=LLaVA-v1.5 7B, Parameter Ratio (rho1)=100%, Precision=FP162026.05 | 68.01 | 64.1 | |
| IO-SVDModel=LLaVA1.5 7B, Ratio=90%2026.05 | 67.92 | — | |
| QSVDModel=LLaVA1.5 7B, Ratio=90%2026.05 | 67.72 | — | |
| QSVD-noQModel Architecture=LLaVA-v1.5 7B, Parameter Ratio (rho1)=90%, Precision=FP162026.05 | 67.72 | 62.64 | |
| WSVD-noQModel Architecture=LLaVA-v1.5 7B, Parameter Ratio (rho1)=80%, Precision=FP162026.05 | 67.72 | 63.54 | |
| WSVD-noQModel Architecture=LLaVA-v1.5 7B, Parameter Ratio (rho1)=70%, Precision=FP162026.05 | 67.28 | 63.54 | |
| Qwen-VLLLM=Qwen-7B, Res.=4482024.03 | 67.1 | — | |
| Qwen-VLLLM=Qwen-7B, Resolution=4482024.11 | 67.1 | — | |
| Qwen-VLLLM=Qwen-7B, Activated Parameters=6.7B, Resolution=4482024.01 | 67.1 | — | |
| QSVD-noQModel Architecture=LLaVA-v1.5 7B, Parameter Ratio (rho1)=70%, Precision=FP162026.05 | 67.08 | 62.64 | |
| LASER-noQModel Architecture=LLaVA-v1.5 7B, Parameter Ratio (rho1)=60%, Precision=FP162026.05 | 67.08 | 63.81 | |
| LASER-noQModel Architecture=LLaVA-v1.5 7B, Parameter Ratio (rho1)=70%, Precision=FP162026.05 | 67.03 | 63.81 | |
| LLaVA1.5Source=CVPR’242024.07 | 66.8 | — | |
| LLaVA-v1.5LLM=Vicuna-7B, Res.=3362024.03 | 66.8 | — | |
| LLaVA-1.5LLM=Vicuna-7B, Resolution=3362024.11 | 66.8 | — | |
| LLaVA-1.5LLM=Vicuna-7B, Activated Parameters=6.7B, Resolution=3362024.01 | 66.8 | — | |
| IO-SVDModel=LLaVA1.5 7B, Ratio=60%2026.05 | 66.73 | — | |
| LASER-noQModel Architecture=LLaVA-v1.5 7B, Parameter Ratio (rho1)=50%, Precision=FP162026.05 | 66.73 | 63.81 | |
| IO-SVDModel=LLaVA1.5 7B, Ratio=50%2026.05 | 65.99 | — | |
| WSVD-noQModel Architecture=LLaVA-v1.5 7B, Parameter Ratio (rho1)=60%, Precision=FP162026.05 | 65.89 | 63.54 | |
| Emu2-ChatLLM=LLAMA-33B, Res.=4482024.03 | 65.5 | — | |
| WSVD-noQModel Architecture=LLaVA-v1.5 7B, Parameter Ratio (rho1)=50%, Precision=FP162026.05 | 65.49 | 63.54 | |
| SVD-LLMModel=LLaVA1.5 7B, Ratio=90%2026.05 | 65.44 | — | |
| SVD-LLMModel Architecture=LLaVA-v1.5 7B, Parameter Ratio (rho1)=90%, Precision=FP162026.05 | 65.44 | 58.47 |