Visual Question Answering on ScienceQA (SQAI)
74.2AccuracyLLaVA-1.5 + MMInstruct
Evaluation Results
| Method | Links | |
|---|---|---|
| LLaVA-1.5 + MMInstructLLM=Vicuna-13B2024.07 | 74.2 | |
| LLaVA-VTLLM=Vicuna-13B, #PT=558K, #IT=665K, Representation=E + T2024.03 | 72.6 | |
| LLaVA-CapLLM=Vicuna-13B, #PT=558K, #IT=665K, Representation=E + T2024.03 | 71.9 | |
| LLaVA-SGLLM=Vicuna-13B, #PT=558K, #IT=665K, Representation=E + T2024.03 | 71.8 | |
| LLaVA-1.5 (V-13B)LLM=Vicuna-13B, #PT=558K, #IT=665K, Representation=Visual Embeddings (E)2024.03 | 71.6 | |
| LLaVA-1.5LLM=Vicuna-13B2024.07 | 71.6 | |
| LLaVA-DCapLLM=Vicuna-13B, #PT=558K, #IT=665K, Representation=E + T2024.03 | 71.2 | |
| LLaVA-1.5 + MMInstructLLM=Vicuna-7B2024.07 | 71.2 | |
| Vicuna-VTLLM=Vicuna-13B, #IT=665K, Representation=Text (T)2024.03 | 70 | |
| Vicuna-CapLLM=Vicuna-13B, #IT=665K, Representation=Text (T)2024.03 | 69.5 | |
| γ-MoD-LLaVA-HRParam.=13B, Speed=4.82024.10 | 69.5 | |
| Vicuna-DCapLLM=Vicuna-13B, #IT=665K, Representation=Text (T)2024.03 | 69.3 | |
| Vicuna-SGLLM=Vicuna-13B, #IT=665K, Representation=Text (T)2024.03 | 68.9 | |
| MoE-LLaVAParam.=5B, Speed=5.62024.10 | 68.5 | |
| LLaVA-SPLLM=Vicuna-7B, Resolution=3362026.07 | 68.4 | |
| Qwen-VL-ChatLLM=Qwen-7B, #PT=1.4B, #IT=50M, Representation=Visual Embeddings (E)2024.03 | 68.2 | |
| VILAParam.=7B2024.10 | 68.2 | |
| Qwen-VL-chatLLM=Qwen-7B2024.07 | 68.2 | |
| Qwen-VL-ChatLLM=Qwen-7B, Resolution=4482026.07 | 68.2 | |
| LLaVA-HRParam.=13B, Speed=3.12024.10 | 68.1 | |
| LLaVA-HRParam.=7B, Speed=4.72024.10 | 67.9 | |
| γ-MoD-LLaVA-HRParam.=7B, Speed=7.22024.10 | 67.9 | |
| LLaVA-1.5LLM=Vicuna-7B, Resolution=3362026.07 | 67.6 | |
| Qwen-VLLLM=Qwen-7B, #PT=1.4B, #IT=50M, Representation=Visual Embeddings (E)2024.03 | 67.1 | |
| Qwen-VLParam.=10B, Speed=4.62024.10 | 67.1 | |
| Qwen-VLLLM=Qwen-7B2024.07 | 67.1 | |
| Qwen-VLLLM=Qwen-7B, Resolution=4482026.07 | 67.1 | |
| LLaVA-1.5 (V-7B)LLM=Vicuna-7B, #PT=558K, #IT=665K, Representation=Visual Embeddings (E)2024.03 | 66.8 | |
| LLaVA-1.5Param.=7B, Speed=8.12024.10 | 66.8 | |
| LLaVA-1.5LLM=Vicuna-7B2024.07 | 66.8 | |
| LLaVA-1.5LLM=Vicuna-7B, Resolution=3362026.07 | 66.8 | |
| γ-MoD-LLaVAParam.=7B, Speed=10.32024.10 | 64.7 | |
| InstructBLIP (V-13B)LLM=Vicuna-13B, #PT=129M, #IT=1.2M, Representation=Visual Embeddings (E)2024.03 | 63.1 | |
| InstructBLIPParam.=14B2024.10 | 63.1 | |
| InstructBLIPLLM=Vicuna-13B2024.07 | 63.1 | |
| MoE-LLaVAParam.=3B, Speed=8.52024.10 | 62.6 | |
| TrustLLaVALLM=Vicuna-7B, Resolution=3362026.07 | 62.1 | |
| BLIP-2LLM=Vicuna-13B, #PT=129M, Representation=Visual Embeddings (E)2024.03 | 61 | |
| BLIP-2LLM=Vicuna-13B2024.07 | 61 | |
| InstructBLIP (V-7B)LLM=Vicuna-7B, #PT=129M, #IT=1.2M, Representation=Visual Embeddings (E)2024.03 | 60.5 | |
| InstructBLIPLLM=Vicuna-7B2024.07 | 60.5 | |
| TrustLLaVA (MLP)LLM=Vicuna-7B, Resolution=3362026.07 | 60.5 |