Visual Question Answering on VQAv2 (test-dev)
86.1AccuracyPaLI-X-55B
Evaluation Results
| Method | Links | |
|---|---|---|
| PaLI-X-55BModel Type=Specialist SOTA2023.10 | 86.1 | |
| PaLI-XCategory=Specialist2023.12 | 86 | |
| NVILA# Parameters=8B2024.12 | 85.4 | |
| NVLM-D-1.0# Parameters=78B2024.12 | 85.4 | |
| LLaVA-OneVision# Parameters=72B2024.12 | 85.2 | |
| SMOLA-PALI-XFTCategory=Generalist, Fine-tuned=true2023.12 | 85 | |
| NVILA-Lite# Parameters=8B2024.12 | 85 | |
| NVILA# Parameters=15B2024.12 | 84.8 | |
| CogVLMCategory=Specialist2023.12 | 84.7 | |
| Florence-VL# Parameters=8B2024.12 | 84.7 | |
| PaLI-XFTCategory=Generalist, Fine-tuned=true2023.12 | 84.4 | |
| VILA-1.5# Parameters=40B2024.12 | 84.3 | |
| BEIT-3Category=Specialist2023.12 | 84.2 | |
| LLaVA-OneVision# Parameters=8B2024.12 | 84 | |
| Cambrian-1# Parameters=34B2024.12 | 83.8 | |
| NVILA-Lite# Parameters=15B2024.12 | 83.7 | |
| LLaVA-NeXT# Parameters=34B2024.12 | 83.7 | |
| SMOLA48-PaLI-3FTCategory=Generalist, Fine-tuned=true2023.12 | 83.6 | |
| CogVLMCategory=Generalist2023.12 | 83.4 | |
| VILA-1.5# Parameters=8B2024.12 | 83 | |
| Qwen2-VL# Parameters=8B2024.12 | 82.9 | |
| VILA-1.5# Parameters=13B2024.12 | 82.8 | |
| PaLI-3FTCategory=Generalist, Fine-tuned=true2023.12 | 82.5 | |
| LLaVA-1.5-HDLLM=Vicuna-13B, Image Size=448^2, Pretrain Sample Size=558K, Finetune Sample Size=665K2023.10 | 81.8 | |
| GIT2Category=Specialist2023.12 | 81.7 | |
| InternVL-ChatLLM=Vicuna-13B, Res.=336, glue layer=QLLAMA2023.12 | 81.2 | |
| Cambrian-1# Parameters=8B2024.12 | 81.2 | |
| Cambrian-1# Parameters=13B2024.12 | 81.2 | |
| LOVA³-7BLLM=Vicuna-7B2024.05 | 80.3 | |
| InternVL-ChatLLM=Vicuna-13B, Res.=336, glue layer=MLP2023.12 | 80.2 | |
| Gemini 1.5 Pro2024.12 | 80.2 | |
| Pixtral# Parameters=12B2024.12 | 80.2 | |
| LLaVA-1.5LLM=Vicuna-13B, Res.=3362023.12 | 80 | |
| PaLM-ECategory=Specialist2023.12 | 80 | |
| LLaVA-1.5LLM=Vicuna-13B, Image Size=336^2, Pretrain Sample Size=558K, Finetune Sample Size=665K2023.10 | 80 | |
| LLaVA-1.5# Parameters=13B2024.12 | 80 | |
| LLaVA 1.52023.12 | 80 | |
| RLHF-VBackbone=Vicuna v1.0 13B, Visual Module=BEiT-3, Resolution=4482023.12 | 80 | |
| Qwen-VLCategory=Generalist2023.12 | 79.5 | |
| Qwen-VL-Chat2023.12 | 79.5 | |
| InternVL-ChatLLM=Vicuna-7B, Res.=336, glue layer=MLP2023.12 | 79.3 | |
| Qwen-VLLLM=Qwen-7B2023.12 | 78.8 | |
| Qwen-VLLLM=Qwen-7B, Image Size=448^2, Pretrain Sample Size=1.4B+, Finetune Sample Size=50M+2023.10 | 78.8 | |
| GPT-4o2024.12 | 78.7 | |
| LLaVA-1.5LLM=Vicuna-7B, Res.=3362023.12 | 78.5 | |
| LLaVA-1.5LLM=Vicuna-7B, Image Size=336^2, Pretrain Sample Size=558K, Finetune Sample Size=665K2023.10 | 78.5 | |
| LLaVA-1.5LLM=Vicuna-7B2024.05 | 78.5 | |
| LLaVA-1.5# Parameters=7B2024.12 | 78.5 | |
| Qwen-VL-ChatLLM=Qwen-7B2023.12 | 78.2 | |
| Qwen-VL-ChatLLM=Qwen-7B, Image Size=448^2, Pretrain Sample Size=1.4B+, Finetune Sample Size=50M+2023.10 | 78.2 | |
| Unified-IOCategory=Generalist2023.12 | 77.9 | |
| ShikraLLM=Vicuna-13B2023.12 | 77.4 | |
| ShikraLLM=Vicuna-13B, Image Size=224^2, Pretrain Sample Size=600K, Finetune Sample Size=5.5M2023.10 | 77.4 | |
| UncompressedModel=BLIP, Pruning Mode=/, Pruning Ratio=/, GFLOPs=186.12026.04 | 77.4 | |
| GPT-4VZero-shot=true2023.12 | 77.2 | |
| InternVL2# Parameters=8B2024.12 | 76.7 | |
| CoMPModel=BLIP, Pruning Mode=C, Pruning Ratio=0.7, GFLOPs=59.72026.04 | 76.5 | |
| MADTPModel=BLIP, Pruning Mode=T, Pruning Ratio=0.7, GFLOPs=61.62026.04 | 76.3 | |
| LOVA³-1.5BLLM=Phi-1.5B2024.05 | 75.8 | |
| UPopModel=BLIP, Pruning Mode=P, Pruning Ratio=0.7, GFLOPs=62.32026.04 | 74.5 | |
| LLaVA-Phi-1.5LLM=Phi-1.5B2024.05 | 73.2 | |
| InternVL-ChatLLM=Vicuna-7B, Res.=224, glue layer=QLLAMA2023.12 | 72.3 | |
| InternVL-ChatLLM=Vicuna-13B, Res.=224, glue layer=QLLAMA2023.12 | 71.7 | |
| Claude 3.5 Sonnet2024.12 | 70.7 | |
| MH-MoEExperts Number=82024.04 | 70.1 | |
| X-MoEExperts Number=82024.04 | 68.4 | |
| Dense2024.04 | 65.9 | |
| IDEFICS-80BLLM=LLaMA-65B2023.12 | 65 | |
| BLIP-2LLM=Vicuna-13B, Image Size=224^2, Pretrain Sample Size=129M, Finetune Sample Size=None2023.10 | 65 | |
| IDEFICS-80BLLM=LLaMA-65B, Image Size=224^2, Pretrain Sample Size=353M, Finetune Sample Size=1M2023.10 | 60 | |
| BLIP-2LLM=Vicuna-7b, Data=104M, steps=330k, Zero-shot=true2023.10 | 57.8 | |
| EVLGenLLM=Vicuna-7b, Data=104M, steps=150k, Zero-shot=true2023.10 | 55.5 | |
| EVLGenLLM=Vicuna-7b, Data=104M, steps=250k, Zero-shot=true2023.10 | 54.8 | |
| EVLGenLLM=Vicuna-7b, Data=11M, steps=150k, Zero-shot=true2023.10 | 54.6 | |
| EVLGenLLM=Vicuna-7b, Data=104M, steps=90k, Zero-shot=true2023.10 | 53.4 | |
| IDEFICS-9BLLM=LLaMA-7B, Image Size=224^2, Pretrain Sample Size=353M, Finetune Sample Size=1M2023.10 | 50.9 | |
| InstructBLIPLLM=Vicuna-13B2023.12 | 49.5 | |
| InstructBLIPLLM=Vicuna-7B2023.12 | 49.2 | |
| BLIP-2LLM=Vicuna-13B2023.12 | 41 | |
| IDEFICS-80B-ILLM=LLaMA-65B2023.12 | 37.4 |