Multimodal Question Answering on SEED-Bench
71.1Accuracy (All)QMoSLoRA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| QMoSLoRAModel=InternLM2 + ViT, Initialization=Orthogonal, Quantization=4-bit2024.06 | 71.1 | — | — | |
| QLoRAModel=InternLM2 + ViT, Quantization=4-bit2024.06 | 70.4 | — | — | |
| QMoSLoRAModel=InternLM2 + ViT, Initialization=Kaiming uniform, Quantization=4-bit2024.06 | 70.3 | — | — | |
| MoSLoRAModel=LLaMA-3 + ViT, Initialization=Orthogonal2024.06 | 69 | — | — | |
| MoSLoRAModel=LLaMA-3 + ViT, Initialization=Kaiming uniform2024.06 | 68.9 | — | — | |
| LoRAModel=LLaMA-3 + ViT2024.06 | 68.8 | — | — | |
| LLaVA-1.5-13B-HDLLM=Vicuna-13B, Image Size=448x448, Pre-train Sample Size=558K, Fine-tune Sample Size=665K2023.10 | 62.6 | — | — | |
| LLaVA-1.5-13BLLM=Vicuna-13B, Image Size=336x336, Pre-train Sample Size=558K, Fine-tune Sample Size=665K2023.10 | 61.6 | — | — | |
| LLaVA-1.5-7BLLM=Vicuna-7B, Image Size=336x336, Pre-train Sample Size=558K, Fine-tune Sample Size=665K2023.10 | 58.6 | — | — | |
| Qwen-VL-Chat2023.08 | 58.2 | 65.4 | 37.8 | |
| Qwen-VL-ChatLLM=Qwen-7B, Image Size=448x448, Pre-train Sample Size=1.4B*, Fine-tune Sample Size=50M+2023.10 | 58.2 | — | — | |
| Qwen-VL2023.08 | 56.3 | 62.3 | 39.1 | |
| Qwen-VLLLM=Qwen-7B, Image Size=448x448, Pre-train Sample Size=1.4B+, Fine-tune Sample Size=50M+2023.10 | 56.3 | — | — | |
| InstructBLIP2023.08 | 53.4 | 58.8 | 38.1 | |
| InstructBLIP-8BLLM=Vicuna-7B, Image Size=224x224, Pre-train Sample Size=129M, Fine-tune Sample Size=1.2M2023.10 | 53.4 | — | — | |
| BLIP2-14BLLM=Vicuna-13B, Image Size=224x224, Pre-train Sample Size=129M2023.10 | 46.4 | — | — | |
| MiniGPT42023.08 | 42.8 | 47.4 | 29.9 | |
| mPLUG-Owl2023.08 | 34 | 37.9 | 23 | |
| LLAVA2023.08 | 33.5 | 37 | 23.8 | |
| LLaVA-7BLLM=Vicuna-7B2023.10 | 33.5 | — | — | |
| LLaMA-AdapterV22023.08 | 32.7 | 35.2 | 25.8 |