Visual Question Answering on GQA balanced (test-dev)
77.4AccuracyShikra-13B
Evaluation results
| Method | Links | |
|---|---|---|
| Shikra-13BSize=> 8B2024.02 | 77.4 | |
| CFRtrained on QA=true2024.03 | 72.1 | |
| LLaVA-NeXT-13BSize=> 8B2024.02 | 65.4 | |
| ShareGPT4V-13BSize=> 8B2024.02 | 64.8 | |
| VILA1.5-13BSize=> 8B2024.02 | 64.3 | |
| LLaVA-NeXT-7BSize=4B < Size < 8B2024.02 | 64.2 | |
| SVIT-v1.5-13BSize=> 8B2024.02 | 64.1 | |
| Bunny-8BSize=4B < Size < 8B2024.02 | 64 | |
| Mipha-3BSize=Size < 4B2024.02 | 63.9 | |
| LVIS-INSTRUCT4V-13BSize=> 8B2024.02 | 63.6 | |
| Imp-v1.5-4B-Phi3Size=Size < 4B2024.02 | 63.5 | |
| LLaVA-v1.5-13B (LORA)Size=> 8B2024.02 | 63.3 | |
| Bunny-4BSize=Size < 4B2024.02 | 63.2 | |
| LLaVA-v1.5-7B (LORA)Size=4B < Size < 8B2024.02 | 63 | |
| NSMtrained on QA=true2024.03 | 63 | |
| TinyLLaVA-Phi-2-SigLIP-3.1BSize=Size < 4B2024.02 | 62.1 | |
| VILA1.5-3BSize=Size < 4B2024.02 | 61.5 | |
| MobileVLM-V2-3BSize=Size < 4B2024.02 | 61.1 | |
| MiniGPT-v2Size=4B < Size < 8B2024.02 | 60.3 | |
| LXMERTtrained on QA=true2024.03 | 60 | |
| SPHINX-Intern2Size=4B < Size < 8B2024.02 | 56.2 | |
| mPLUG-Owl2Size=4B < Size < 8B2024.02 | 56.1 | |
| LGCNtrained on QA=true2024.03 | 55.8 | |
| InstructBLIP-Vicuna-13BSize=> 8B2024.02 | 49.5 | |
| InstructBLIPzero-shot=true2024.03 | 49.5 | |
| InstructBLIP-Vicuna-7BSize=4B < Size < 8B2024.02 | 49.2 | |
| FlexCap-LLMzero-shot=true2024.03 | 48.8 | |
| ViperGPTzero-shot=true2024.03 | 48.1 | |
| InstructBLIP-Flan-T5-XXLSize=> 8B2024.02 | 47.9 | |
| BLIPv2zero-shot=true2024.03 | 44.7 | |
| BLIP-2-Flan-T5-XXLSize=> 8B2024.02 | 44.6 | |
| BLIP-2-Vicuna-13BSize=> 8B2024.02 | 41 |