Science Question Answering on ScienceQA
99.8AccuracyPhi-4 pass@N (Upper Bound)
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Phi-4 pass@N (Upper Bound)# Sample=-2025.11 | 99.8 | — | — | — | — | — | — | |
| LongVILA-7B (S3)LLM=Qwen2-7B, Resolution=dynamic2024.08 | 98.5 | — | — | — | — | — | — | |
| RLHFlow-PRM-Deepseek-Data# Sample=253K2025.11 | 98 | — | — | — | — | — | — | |
| VanillaBackbone=InternVL-2.5-8B, Token Reduction Ratio=0%2026.06 | 98 | — | — | — | — | — | — | |
| Qwen2.5-Math-7B-PRM800k# Sample=263K2025.11 | 97.5 | — | — | — | — | — | — | |
| Qwen2.5-Math-7B-PRM# Sample=860K2025.11 | 97.5 | — | — | — | — | — | — | |
| GPT-5-highModel Category=Proprietary Models2026.06 | 97.4 | — | — | — | — | — | — | |
| RLHFlow-PRM-Mistral-Data# Sample=273K2025.11 | 97.3 | — | — | — | — | — | — | |
| Universal-PRM-Qwen2.5-Math-7B# Sample=690K2025.11 | 96.8 | — | — | — | — | — | — | |
| Qwen3-VL-32BToken Ratio=100%2026.05 | 96.72 | — | — | — | — | — | — | |
| Skywork-PRM-1.5B# Sample=Unk2025.11 | 96.6 | — | — | — | — | — | — | |
| Math-Shepherd-PRM-7B# Sample=440K2025.11 | 96.6 | — | — | — | — | — | — | |
| CLSE localBackbone=InternVL-2.5-8B, Token Reduction Ratio=66.7%, FFT Type=Local2026.06 | 96.6 | — | — | — | — | — | — | |
| CLSE globalBackbone=InternVL-2.5-8B, Token Reduction Ratio=66.7%, FFT Type=Global2026.06 | 96.5 | — | — | — | — | — | — | |
| H4-Qwen2.5-PRM-1.5B-0.2# Sample=369K2025.11 | 96.4 | — | — | — | — | — | — | |
| FastVBackbone=InternVL-2.5-8B, Token Reduction Ratio=66.7%2026.06 | 96.4 | — | — | — | — | — | — | |
| CLSE localBackbone=InternVL-2.5-8B, Token Reduction Ratio=77.8%, FFT Type=Local2026.06 | 96.4 | — | — | — | — | — | — | |
| CLSE globalBackbone=InternVL-2.5-8B, Token Reduction Ratio=77.8%, FFT Type=Global2026.06 | 96.4 | — | — | — | — | — | — | |
| MiniCPM-v2.6 + HIPPOBackbone=MiniCPM-v2.6, Alignment Strategy=HIPPO2026.02 | 96.33 | — | — | — | — | — | — | |
| InternVL3.5-38BRatio=100%, Backbone=InternVL3.5-38B2026.05 | 96.2 | — | — | — | — | — | — | |
| Gemini-2.5-ProModel Category=Proprietary Models2026.06 | 96.2 | — | — | — | — | — | — | |
| CLSE localBackbone=InternVL-2.5-8B, Token Reduction Ratio=88.9%, FFT Type=Local2026.06 | 95.8 | — | — | — | — | — | — | |
| Phi-4 pass@1 (Lower Bound)# Sample=-2025.11 | 95.7 | — | — | — | — | — | — | |
| ReProbe, Attn+Logit, Qwen3-8B-anno# Sample=32K2025.11 | 95.7 | — | — | — | — | — | — | |
| FastVBackbone=InternVL-2.5-8B, Token Reduction Ratio=77.8%2026.06 | 95.7 | — | — | — | — | — | — | |
| CLSE globalBackbone=InternVL-2.5-8B, Token Reduction Ratio=88.9%, FFT Type=Global2026.06 | 95.7 | — | — | — | — | — | — | |
| TVI-CoTModel Category=MLLM-based Chain-of-Thought Methods, Model Scale=8B2026.06 | 95.2 | — | — | — | — | — | — | |
| MiniCPM-v2.6Backbone=MiniCPM-v2.62026.02 | 95.19 | — | — | — | — | — | — | |
| Qwen3-VL + DISCO (10K)Backbone=Qwen3-VL, Alignment Strategy=DISCO (10K)2026.02 | 95.09 | — | — | — | — | — | — | |
| VisionZipRatio=60%, Backbone=InternVL3.5-38B2026.05 | 95.07 | — | — | — | — | — | — | |
| LLaVA-OneVision-1.5-8BModel Category=Open-Source MLLMs, Model Scale=8B2026.06 | 95 | — | — | — | — | — | — | |
| F3AToken Ratio=60%2026.05 | 94.97 | — | — | — | — | — | — | |
| Qwen3-VL + Textual (10K)Backbone=Qwen3-VL, Alignment Strategy=Textual (10K)2026.02 | 94.94 | — | — | — | — | — | — | |
| F3ARatio=60%, Backbone=InternVL3.5-38B2026.05 | 94.85 | — | — | — | — | — | — | |
| Qwen3-VLBackbone=Qwen3-VL2026.02 | 94.79 | — | — | — | — | — | — | |
| Majority Voting# Sample=-2025.11 | 94.6 | — | — | — | — | — | — | |
| VanillaAverage Token Reduction=0.0%, Backbone=Qwen3-VL-8B2026.06 | 94.5 | — | — | — | — | — | — | |
| HiP-LoRABackbone=Llama-3.1-8B, Rank (r)=16, k (spectral budget)=32, Number of seeds=32026.04 | 94.05 | — | — | — | — | — | — | |
| HiP-LoRA (k=32, r=16)Model=Llama-3.1-8B, Merge Status=Before2026.04 | 94.05 | — | — | — | — | — | — | |
| FastVRatio=60%, Backbone=InternVL3.5-38B2026.05 | 93.8 | — | — | — | — | — | — | |
| DivPruneRatio=60%, Backbone=InternVL3.5-38B2026.05 | 93.8 | — | — | — | — | — | — | |
| PiSSABackbone=Llama-3.1-8B, Rank (r)=16, Number of seeds=32026.04 | 93.62 | — | — | — | — | — | — | |
| PiSSA (r=16)Model=Llama-3.1-8B, Merge Status=Before2026.04 | 93.62 | — | — | — | — | — | — | |
| FastVToken Ratio=60%2026.05 | 93.54 | — | — | — | — | — | — | |
| CDPrunerToken Ratio=60%2026.05 | 93.43 | — | — | — | — | — | — | |
| CDPrunerRatio=60%, Backbone=InternVL3.5-38B2026.05 | 93.41 | — | — | — | — | — | — | |
| DivPruneRatio=40%, Backbone=InternVL3.5-38B2026.05 | 93.12 | — | — | — | — | — | — | |
| F3ARatio=40%, Backbone=InternVL3.5-38B2026.05 | 93.12 | — | — | — | — | — | — | |
| FastVLM-1.7BVision Encoder=FastViTHD, LLM=Qw.2, #Visual Tokens=256, Vis. Enc. Size (M)=125, Evaluation Library=lmms-eval2024.12 | 92.9 | — | — | — | — | — | — | |
| Full PrecModel=Qwen2.5-VL-32B-Instruct, Bit-width=Full Precision2025.08 | 92.69 | — | — | — | — | — | — | |
| FastVRatio=40%, Backbone=InternVL3.5-38B2026.05 | 92.54 | — | — | — | — | — | — | |
| Qwen3-VL-8B-ThinkingStrategy=LongCoT2026.02 | 92.5 | — | — | — | — | — | — | |
| FastVBackbone=InternVL-2.5-8B, Token Reduction Ratio=88.9%2026.06 | 92.5 | — | — | — | — | — | — | |
| LoRABackbone=Llama-3.1-8B, Rank (r)=16, Number of seeds=32026.04 | 92.26 | — | — | — | — | — | — | |
| LoRA(r=16)Model=Llama-3.1-8B, Merge Status=Before2026.04 | 92.26 | — | — | — | — | — | — | |
| SAPStrategy=SAP2026.02 | 92.2 | — | — | — | — | — | — | |
| F3AToken Ratio=40%2026.05 | 92.1 | — | — | — | — | — | — | |
| Proj-LoRABackbone=Llama-3.1-8B, Rank (r)=16, Number of seeds=32026.04 | 92.08 | — | — | — | — | — | — | |
| Proj-LoRA(r=16)Model=Llama-3.1-8B, Merge Status=Before2026.04 | 92.08 | — | — | — | — | — | — | |
| Split-LoRA(4×4)Model=Llama-3.1-8B, Merge Status=Before2026.04 | 92 | — | — | — | — | — | — | |
| VAPO-Thinker-7BModel Category=MLLM-based Chain-of-Thought Methods, Model Scale=7B2026.06 | 92 | — | — | — | — | — | — | |
| VisionZipRatio=40%, Backbone=InternVL3.5-38B2026.05 | 91.94 | — | — | — | — | — | — | |
| Qwen3-VL-8B (Baseline)Model Category=Baseline, Model Scale=8B2026.06 | 91.8 | — | — | — | — | — | — | |
| FP16Spiking=false, Time Step=N/A, Backbone=Qwen2VL-72B2026.04 | 91.67 | — | — | — | — | — | — | |
| VisionZipToken Ratio=60%2026.05 | 91.48 | — | — | — | — | — | — | |
| MODIXSize=8B, Backbone=InternVL3.52026.04 | 91.13 | — | — | — | — | — | — | |
| CDPrunerRatio=40%, Backbone=InternVL3.5-38B2026.05 | 91.1 | — | — | — | — | — | — | |
| SID-vModel=GLM4.1V2025.10 | 91 | — | — | — | — | — | — | |
| SID-cModel=GLM4.1V2025.10 | 91 | — | — | — | — | — | — | |
| CDPrunerToken Ratio=40%2026.05 | 90.97 | — | — | — | — | — | — | |
| DivPruneToken Ratio=60%2026.05 | 90.92 | — | — | — | — | — | — | |
| VanillaBackbone=Qwen3-VL-4B-FP8, Flops Ratio Reduction=0%, Precision=FP82026.04 | 90.8 | — | — | — | — | — | — | |
| ResDecBase Model=Qwen2.5-VL2026.02 | 90.48 | — | — | — | — | — | — | |
| OPERABase Model=Qwen2.5-VL2026.02 | 90.33 | — | — | — | — | — | — | |
| VISTABase Model=Qwen2.5-VL2026.02 | 90.23 | — | — | — | — | — | — | |
| MODIXSize=8B, Backbone=Qwen3-VL2026.04 | 90.16 | — | — | — | — | — | — | |
| VCDBase Model=Qwen2.5-VL2026.02 | 90.08 | — | — | — | — | — | — | |
| MADModel=GLM4.1V2025.10 | 90 | — | — | — | — | — | — | |
| Full PrecModel Backbone=LLaVA-OneVision-7B, Quantization Method=Full Precision, Quantization Bit-width/Group-size=None, Zero-shot=true2025.08 | 89.98 | — | — | — | — | — | — | |
| Full PrecModel=LLaVA-OneVision-7B, Bit-width=Full Precision2025.08 | 89.98 | — | — | — | — | — | — | |
| SpikeMLLM (QuaRot+MSTS+TC-LIF)Spiking=true, Time Step=3/4, Backbone=Qwen2VL-72B2026.04 | 89.84 | — | — | — | — | — | — | |
| InternVL3.5Size=8B, Configuration=Baseline2026.04 | 89.7 | — | — | — | — | — | — | |
| F3ARatio=20%, Backbone=InternVL3.5-38B2026.05 | 89.7 | — | — | — | — | — | — | |
| SmolVLM2-2.2BVision Encoder=ViT-SO400M, LLM=SmolLM2, #Visual Tokens=2106*, Vis. Enc. Size (M)=430, Evaluation Library=lmms-eval2024.12 | 89.6 | — | — | — | — | — | — | |
| Qwen2.5-VL-7BRatio=100%, Backbone=Qwen2.5-VL-7B2026.05 | 89.12 | — | — | — | — | — | — | |
| InternVL3.5-8BRatio=100%, Backbone=InternVL3.5-8B2026.05 | 89 | — | — | — | — | — | — | |
| RegularBase Model=Qwen2.5-VL2026.02 | 88.99 | — | — | — | — | — | — | |
| FastVToken Ratio=40%2026.05 | 88.92 | — | — | — | — | — | — | |
| GPTQModel Backbone=LLaVA-OneVision-7B, Quantization Method=GPTQ, Quantization Bit-width/Group-size=INT3g128, Zero-shot=true2025.08 | 88.85 | — | — | — | — | — | — | |
| FastVRatio=20%, Backbone=InternVL3.5-38B2026.05 | 88.81 | — | — | — | — | — | — | |
| VisionZipToken Ratio=40%2026.05 | 88.81 | — | — | — | — | — | — | |
| DoLaBase Model=Qwen2.5-VL2026.02 | 88.75 | — | — | — | — | — | — | |
| VanillaBackbone=Qwen2-VL-7B, Flops Ratio Reduction=0%2026.04 | 88.7 | — | — | — | — | — | — | |
| VLMQModel Backbone=LLaVA-OneVision-7B, Quantization Method=VLMQ, Quantization Bit-width/Group-size=INT3g128, Zero-shot=true2025.08 | 88.42 | — | — | — | — | — | — | |
| Qwen3-VLSize=8B, Configuration=Baseline2026.04 | 88.41 | — | — | — | — | — | — | |
| CDPrunerRatio=20%, Backbone=InternVL3.5-38B2026.05 | 88.41 | — | — | — | — | — | — | |
| DivPruneRatio=20%, Backbone=InternVL3.5-38B2026.05 | 88.41 | — | — | — | — | — | — | |
| FP16Backbone=Qwen2.5-VL-7B, Bits=W16A162026.05 | 88.4 | — | — | — | — | — | — | |
| SpikeMLLM(SQ)Spiking=true, Time Step=255, Backbone=Qwen2VL-72B2026.04 | 88.3 | — | — | — | — | — | — | |
| Full PrecModel Backbone=Qwen2.5-VL-7B-Instruct, Quantization Method=Full Precision, Quantization Bit-width/Group-size=None, Zero-shot=true2025.08 | 88.26 | — | — | — | — | — | — |