Multimodal Question Answering on ScienceQA (test)
92.53AccuracyLLaVa + GPT-4 (judge)
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| LLaVa + GPT-4 (judge)#Params=13B, Synthetic instruction finetuning / GPT-support (♠)=true2023.07 | 92.53 | — | 91.56 | 96.74 | 91.09 | 90.62 | 88.99 | 93.52 | 92.73 | 92.16 | |
| Multimodal-CoT#Params=770M2023.07 | 91.68 | — | 95.91 | 82 | 90.82 | 95.26 | 88.8 | 92.89 | 92.44 | 90.31 | |
| ViT-L/14-224framework=LLaVA, vision encoder=ViT-L/14-2242023.12 | 91.2 | 88.6 | — | — | — | — | — | — | — | — | |
| LLaVa#Params=13B, Synthetic instruction finetuning / GPT-support (♠)=true2023.07 | 90.92 | — | 90.36 | 95.95 | 88 | 89.49 | 88 | 90.66 | 90.93 | 90.9 | |
| SciTune-ScienceQA (CTOM)#Params=13B2023.07 | 90.03 | — | 89.3 | 95.61 | 87 | 93.08 | 86.67 | 91.75 | 84.37 | 91.3 | |
| SF-LCLIPframework=LLaVA, vision encoder=SF-LCLIP, tokens=1282023.12 | 89.8 | 85.8 | — | — | — | — | — | — | — | — | |
| SF-LCLIPframework=LLaVA, vision encoder=SF-LCLIP, tokens=642023.12 | 89.2 | 84.5 | — | — | — | — | — | — | — | — | |
| Human Average#Params=-2023.07 | 88.4 | — | 90.23 | 84.97 | 87.48 | 89.6 | 87.5 | 88.1 | 91.59 | 82.42 | |
| Chameleon (GPT-4)#Params=1T+, Synthetic instruction finetuning / GPT-support (♠)=true2023.07 | 86.54 | — | 89.83 | 74.13 | 89.82 | 88.27 | 77.64 | 92.13 | 88.03 | 83.72 | |
| SciTune-ScienceQA (CTOM)#Params=7B2023.07 | 86.11 | — | 84.5 | 94.15 | 82.91 | 88.35 | 83.64 | 88.74 | 85.05 | 85.6 | |
| SciTune-ScienceQA (C)#Params=7B2023.07 | 85.61 | — | 84.36 | 92.23 | 82.81 | 89.56 | 81.26 | 88.29 | 81.28 | 86.03 | |
| LLaMA-Adapter#Params=13B, Synthetic instruction finetuning / GPT-support (♠)=true2023.07 | 85.19 | — | 84.37 | 88.3 | 84.36 | 83.72 | 80.32 | 86.9 | 85.83 | 84.05 | |
| Multimodal-CoT#Params=223M2023.07 | 84.91 | — | 87.52 | 77.17 | 85.82 | 87.88 | 82.9 | 86.83 | 84.65 | 85.37 | |
| GPT-4 CoT#Params=1T+, Synthetic instruction finetuning / GPT-support (♠)=true2023.07 | 83.99 | — | 85.48 | 72.44 | 90.27 | 82.65 | 71.49 | 92.89 | 86.66 | 79.04 | |
| Chameleon (ChatGPT)#Params=175B+, Synthetic instruction finetuning / GPT-support (♠)=true2023.07 | 79.93 | — | 81.62 | 70.64 | 84 | 79.77 | 70.8 | 86.62 | 81.86 | 76.53 | |
| LoRAModel=Qwen2.5-VL-7B, lr_llm=1e-52026.02 | 79.84 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=Qwen2.5-VL-7B, r_llm=8, r_ve=best2026.02 | 79.84 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=Qwen2.5-VL-7B, r_llm=16, r_ve=best2026.02 | 79.84 | — | — | — | — | — | — | — | — | — | |
| MARSModel=Qwen2.5-VL-7B2026.02 | 79.64 | — | — | — | — | — | — | — | — | — | |
| MARSModel=Qwen2.5-VL-7B2026.02 | 79.64 | — | — | — | — | — | — | — | — | — | |
| MARSModel=Qwen2.5-VL-3B2026.02 | 79.24 | — | — | — | — | — | — | — | — | — | |
| MARSModel=Qwen2.5-VL-3B2026.02 | 79.24 | — | — | — | — | — | — | — | — | — | |
| ChatGPT CoT#Params=175B+, Synthetic instruction finetuning / GPT-support (♠)=true2023.07 | 78.31 | — | 78.82 | 70.98 | 83.18 | 77.37 | 67.92 | 86.13 | 80.72 | 74.03 | |
| LoRAModel=Qwen2.5-VL-3B, r_llm=8, r_ve=best2026.02 | 78.24 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=Qwen2.5-VL-7B, r_llm=32, r_ve=best2026.02 | 78.24 | — | — | — | — | — | — | — | — | — | |
| CSMRPerception Backbone=Qwen2-VL-7B, Reasoning Backbone=Qwen2-7B, Zero-shot=true2026.05 | 78.2 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=Qwen2.5-VL-3B, lr_llm=1e-52026.02 | 78.04 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=Qwen2.5-VL-3B, r_llm=16, r_ve=best2026.02 | 78.04 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=Qwen2.5-VL-3B, r_llm=32, r_ve=best2026.02 | 78.04 | — | — | — | — | — | — | — | — | — | |
| Full-rank TuningModel=Qwen2.5-VL-7B2026.02 | 77.45 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=Qwen2.5-VL-3B, lr_llm=1e-62026.02 | 76.45 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=Qwen2.5-VL-3B, lr_llm=1e-72026.02 | 76.25 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=Qwen2.5-VL-7B, lr_llm=1e-62026.02 | 76.25 | — | — | — | — | — | — | — | — | — | |
| Full-rank TuningModel=Qwen2.5-VL-3B2026.02 | 75.25 | — | — | — | — | — | — | — | — | — | |
| GPT-3 (CoT) (ALE)#Params=175B, Synthetic instruction finetuning / GPT-support (♠)=true2023.07 | 75.17 | — | 75.44 | 70.87 | 78.09 | 74.68 | 67.43 | 79.93 | 78.23 | 69.68 | |
| MARSModel=LLaVA-OV-7B2026.02 | 74.25 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=Qwen2.5-VL-7B, lr_llm=1e-72026.02 | 74.25 | — | — | — | — | — | — | — | — | — | |
| MARSModel=LLaVA-OV-7B2026.02 | 74.25 | — | — | — | — | — | — | — | — | — | |
| UnifiedQA (CoT)#Params=223M2023.07 | 74.11 | — | 71 | 76.04 | 78.91 | 66.42 | 66.53 | 81.81 | 77.06 | 68.82 | |
| GPT-3 (Zero Shot)#Params=175B, Synthetic instruction finetuning / GPT-support (♠)=true2023.07 | 74.04 | — | 75.04 | 66.59 | 78 | 74.24 | 65.74 | 79.58 | 76.36 | 69.87 | |
| AdaLoRAModel=Qwen2.5-VL-7B2026.02 | 73.85 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=LLaVA-OV-7B, r_llm=32, r_ve=best2026.02 | 73.65 | — | — | — | — | — | — | — | — | — | |
| MARSModel=LLaVA-OV-0.5B2026.02 | 72.85 | — | — | — | — | — | — | — | — | — | |
| MARSModel=LLaVA-OV-0.5B2026.02 | 72.85 | — | — | — | — | — | — | — | — | — | |
| GeoLoRAModel=Qwen2.5-VL-7B2026.02 | 72.85 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=LLaVA-OV-7B, lr_llm=1e-52026.02 | 72.26 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=LLaVA-OV-7B, r_llm=16, r_ve=best2026.02 | 72.26 | — | — | — | — | — | — | — | — | — | |
| DDCoTPerception Backbone=Qwen2-VL-7B, Reasoning Backbone=Qwen2-7B, Zero-shot=true2026.05 | 71.9 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=LLaVA-OV-0.5B, lr_llm=1e-52026.02 | 71.06 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=LLaVA-OV-0.5B, r_llm=16, r_ve=best2026.02 | 71.06 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=LLaVA-OV-0.5B, r_llm=8, r_ve=best2026.02 | 70.46 | — | — | — | — | — | — | — | — | — | |
| Full-rank TuningModel=LLaVA-OV-7B2026.02 | 70.46 | — | — | — | — | — | — | — | — | — | |
| GeoLoRAModel=Qwen2.5-VL-3B2026.02 | 70.46 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=LLaVA-OV-7B, r_llm=8, r_ve=best2026.02 | 70.26 | — | — | — | — | — | — | — | — | — | |
| UnifiedQA#Params=223M2023.07 | 70.12 | — | 68.16 | 69.18 | 74.91 | 63.78 | 61.38 | 77.84 | 72.98 | 65 | |
| AdaLoRAModel=Qwen2.5-VL-3B2026.02 | 70.06 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=LLaVA-OV-7B, lr_llm=1e-62026.02 | 69.86 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=LLaVA-OV-0.5B, r_llm=32, r_ve=best2026.02 | 69.86 | — | — | — | — | — | — | — | — | — | |
| Full-rank TuningModel=LLaVA-OV-0.5B2026.02 | 69.66 | — | — | — | — | — | — | — | — | — | |
| GeoLoRAModel=LLaVA-OV-7B2026.02 | 67.81 | — | — | — | — | — | — | — | — | — | |
| CaptionPerception Backbone=Qwen2-VL-7B, Reasoning Backbone=Qwen2-7B, Zero-shot=true2026.05 | 67.7 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=LLaVA-OV-7B, lr_llm=1e-72026.02 | 67.27 | — | — | — | — | — | — | — | — | — | |
| AdaLoRAModel=LLaVA-OV-7B2026.02 | 66.27 | — | — | — | — | — | — | — | — | — | |
| GeoLoRAModel=LLaVA-OV-0.5B2026.02 | 63.52 | — | — | — | — | — | — | — | — | — | |
| AdaLoRAModel=LLaVA-OV-0.5B2026.02 | 62.28 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=LLaVA-OV-0.5B, lr_llm=1e-62026.02 | 61.88 | — | — | — | — | — | — | — | — | — | |
| LoRAModel=LLaVA-OV-0.5B, lr_llm=1e-72026.02 | 59.28 | — | — | — | — | — | — | — | — | — | |
| ICoTPerception Backbone=Qwen2-VL-7B, Reasoning Backbone=Qwen2-VL-7B, Zero-shot=true2026.05 | 56.8 | — | — | — | — | — | — | — | — | — | |
| CCoTPerception Backbone=Qwen2-VL-7B, Reasoning Backbone=Qwen2-VL-7B, Zero-shot=true2026.05 | 56.4 | — | — | — | — | — | — | — | — | — | |
| No-CoTPerception Backbone=Qwen2-VL-7B, Reasoning Backbone=Qwen2-VL-7B, Zero-shot=true2026.05 | 56.3 | — | — | — | — | — | — | — | — | — | |
| SCAFFOLDPerception Backbone=Qwen2-VL-7B, Reasoning Backbone=Qwen2-VL-7B, Zero-shot=true2026.05 | 53.7 | — | — | — | — | — | — | — | — | — | |
| Multimodal CoTPerception Backbone=Qwen2-VL-7B, Reasoning Backbone=Qwen2-VL-7B, Zero-shot=true2026.05 | 51.3 | — | — | — | — | — | — | — | — | — | |
| Random Chance#Params=-2023.07 | 39.83 | — | 40.28 | 46.13 | 29.25 | 47.45 | 40.08 | 33.66 | 39.35 | 40.67 |