Multimodal Science Question Answering on ScienceQA
88.4Overall Average ScoreHuman
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Human2025.12 | 88.4 | 90.23 | 84.97 | 87.48 | 89.6 | 87.5 | 88.1 | 91.59 | 82.42 | |
| GPT-4o-mini (MemVerse)2025.12 | 85.48 | 85.26 | 81.55 | 89.09 | 83.28 | 78.19 | 91.5 | 88.11 | 80.75 | |
| CoT (GPT-4)#T-Param=1T+2025.12 | 83.99 | 85.48 | 72.44 | 90.27 | 82.65 | 71.49 | 92.89 | 86.66 | 79.04 | |
| HoT-T5-Large#T-Param=738M2025.12 | 83.38 | 84.46 | 79.08 | 84.64 | 82.89 | 75.81 | 88.15 | 83.88 | 82.47 | |
| GPT-42025.12 | 82.69 | 84.06 | 73.45 | 87.36 | 81.87 | 70.75 | 90.73 | 84.69 | 79.1 | |
| Qwen2.5-72B (MemVerse)#T-Param=72B2025.12 | 80.25 | 77.53 | 68.95 | 85.36 | 78.68 | 66.39 | 89.2 | 82.31 | 77.76 | |
| DDCoT#T-Param=175B2025.12 | 80.15 | 80.15 | 76.72 | 82.82 | 78.89 | 72.53 | 85.02 | 82.86 | 75.21 | |
| Chameleon (ChatGPT)#T-Param=175B+2025.12 | 79.93 | 81.62 | 70.64 | 84 | 79.77 | 70.8 | 86.62 | 81.86 | 76.53 | |
| Qwen2.5-72B#T-Param=72B2025.12 | 78.37 | 79.64 | 67.1 | 84.9 | 77.56 | 65 | 87.93 | 80.25 | 74.85 | |
| LaVIN-13B#T-Param=13B2025.12 | 77.54 | — | — | — | — | — | — | — | — | |
| CCOT#T-Param=7B2025.12 | 76.84 | — | — | — | — | — | — | — | — | |
| GPT-4o-mini2025.12 | 76.82 | 77.31 | 73.45 | 86.91 | 74.05 | 66.86 | 87.93 | 83.37 | 71.85 | |
| Qwen2.5-7B (MemVerse)#T-Param=7B2025.12 | 75.62 | 74.51 | 68.5 | 78.73 | 75.92 | 66.19 | 81.95 | 79.7 | 64.73 | |
| CoT (GPT-3)#T-Param=173B2025.12 | 75.17 | 75.44 | 70.87 | 78.09 | 74.68 | 67.43 | 79.93 | 78.23 | 69.68 | |
| Qwen2.5-7B (Visual Genome)#T-Param=7B2025.12 | 75.08 | 76.78 | 67.04 | 78.09 | 74.05 | 66.19 | 79.72 | 78.08 | 69.68 | |
| Qwen2.5-7B#T-Param=7B2025.12 | 74.72 | 76.2 | 67.83 | 77.27 | 74.49 | 65.79 | 79.02 | 77.72 | 69.35 | |
| BLIP-22025.12 | 74.17 | — | — | — | — | — | — | — | — | |
| CoT (UnifiedQA)#T-Param=223M2025.12 | 74.11 | 71 | 76.04 | 78.91 | 66.42 | 66.53 | 81.81 | 77.06 | 68.82 | |
| Qwen2.5-7B (Mmkg)#T-Param=7B2025.12 | 73.47 | 73.98 | 66.37 | 78.18 | 71.65 | 64.3 | 79.65 | 76.51 | 68.03 | |
| GraphVis#T-Param=7B2025.12 | 73.18 | — | — | — | — | — | — | — | — | |
| Vanilla (LLaVA-1.5-7B)Retained Tokens=5762026.01 | 69.5 | — | — | — | — | — | — | — | — | |
| PyramidDropRetained Tokens=1922026.01 | 69.2 | — | — | — | — | — | — | — | — | |
| SparseVLMRetained Tokens=1922026.01 | 69.2 | — | — | — | — | — | — | — | — | |
| VisionZipRetained Tokens=642026.01 | 69 | — | — | — | — | — | — | — | — | |
| VisionZipRetained Tokens=1922026.01 | 68.9 | — | — | — | — | — | — | — | — | |
| VisionZipRetained Tokens=1282026.01 | 68.9 | — | — | — | — | — | — | — | — | |
| ViTCoPRetained Tokens=642026.01 | 68.8 | — | — | — | — | — | — | — | — | |
| ViTCoPRetained Tokens=1922026.01 | 68.4 | — | — | — | — | — | — | — | — | |
| PyramidDropRetained Tokens=1282026.01 | 68.4 | — | — | — | — | — | — | — | — | |
| ViTCoPRetained Tokens=1282026.01 | 68.4 | — | — | — | — | — | — | — | — | |
| FastVRetained Tokens=1922026.01 | 67.2 | — | — | — | — | — | — | — | — | |
| SparseVLMRetained Tokens=1282026.01 | 67.2 | — | — | — | — | — | — | — | — | |
| SparseVLMRetained Tokens=642026.01 | 62.2 | — | — | — | — | — | — | — | — | |
| PyramidDropRetained Tokens=642026.01 | 60.8 | — | — | — | — | — | — | — | — | |
| FastVRetained Tokens=1282026.01 | 60.2 | — | — | — | — | — | — | — | — | |
| FastVRetained Tokens=642026.01 | 51.2 | — | — | — | — | — | — | — | — |