Science Question Answering on ScienceQA SQA^I v1.0 (test)
74.7SQA^I ScoreMobileVLM V2 7B
Evaluation results
| Method | Links | |
|---|---|---|
| MobileVLM V2 7BLLM=Vicuna-7B, #Samples=3.6M2026.03 | 74.7 | |
| ShareGPT4VLLM=Vicuna-7B, #Samples=1.9M2026.03 | 68.4 | |
| LLaVA-1.5LLM=Vicuna-7B, #Samples=1.2M2026.03 | 66.8 | |
| MobileVLM -V2 1.7BLLM=MobileLLaMA 1.4B, #Samples=3.6M2026.03 | 63.2 | |
| MoE-LLaVA-2BLLM=Qwen-1.5-1.8B, #Samples=2.2M2026.03 | 63.1 | |
| Cosine KD w/ Beta-KD (Instance)LLM=MobileLLaMA 1.4B, #Samples=3.6M2026.03 | 63.1 | |
| MoE-LLaVA-1.6BLLM=StableLM-1.6B, #Samples=2.2M2026.03 | 62.6 | |
| Align-KD w/ Beta-KD (Task)LLM=MobileLLaMA 1.4B, #Samples=3.6M2026.03 | 61.4 | |
| Cosine KDLLM=MobileLLaMA 1.4B, #Samples=3.6M2026.03 | 61.3 | |
| MobileVLM 3BLLM=MobileLLaMA 2.7B, #Samples=3.6M2026.03 | 61.2 | |
| Align-KD w/ Beta-KD (Instance)LLM=MobileLLaMA 1.4B, #Samples=3.6M2026.03 | 61.2 | |
| Cosine KD w/ Beta-KD (Task)LLM=MobileLLaMA 1.4B, #Samples=3.6M2026.03 | 61.2 | |
| Align-KDLLM=MobileLLaMA 1.4B, #Samples=3.6M2026.03 | 61 | |
| MobileVLM 1.7BLLM=MobileLLaMA 1.4B, #Samples=3.6M2026.03 | 57.3 | |
| LLAVADILLM=MobileLLaMA 1.4B, #Samples=3.6M2026.03 | 56 |