Multimodal Understanding on MMBench (test)
90.9AccuracyCounting
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CountingTrain (und)=Counting2026.07 | 90.9 | — | — | — | — | |
| BaselineTrain (und)=Baseline2026.07 | 90.8 | — | — | — | — | |
| Spatial RelationTrain (und)=Spatial Relation2026.07 | 90.8 | — | — | — | — | |
| Text recognitionTrain (und)=Text recognition2026.07 | 90.5 | — | — | — | — | |
| MMCTAgentsetup=zero-shot, critic=true2024.05 | 84.2 | — | — | — | — | |
| Mini-Gemini-HDSize=34B, # tokens per image=2880, Zero-shot=true2024.05 | 80.6 | — | — | — | — | |
| MMCTAgentsetup=zero-shot, critic=false2024.05 | 80.21 | — | — | — | — | |
| LLaVA-NeXTSize=34B, # tokens per image=2880, Zero-shot=true2024.05 | 79.3 | — | — | — | — | |
| Bunny-8BSize=4B < Size < 8B2024.02 | 78.1 | — | — | — | — | |
| GPT-4Vsetup=zero-shot2024.05 | 77 | — | — | — | — | |
| GPT-4V-1106Size=> 8B2024.02 | 77 | — | — | — | — | |
| Idefics2Size=8B, # tokens per image=64, Zero-shot=true2024.05 | 76.8 | — | — | — | — | |
| Idefics2Size=8B, # tokens per image=320, Zero-shot=true2024.05 | 76.7 | — | — | — | — | |
| Bunny-4BSize=Size < 4B2024.02 | 75.7 | — | — | — | — | |
| MM1-ChatSize=30B, # tokens per image=720, Zero-shot=true2024.05 | 75.1 | — | — | — | — | |
| PIIP-LLaVAVision Encoder=ConvNeXt-L, CLIP-L, Resolution=1024/336, LLM=Vicuna-7B, Data=2.7M2025.01 | 74.5 | — | — | — | — | |
| InternLM-XCVision Encoder=EVA-G, Resolution=224, LLM=InternLM-7B, Data=1.1B2025.01 | 74.4 | — | — | — | — | |
| Gemini 1.5 Prosetup=zero-shot2024.05 | 73.6 | — | — | — | — | |
| DeepSeek-VLSize=7B, # tokens per image=576, Zero-shot=true2024.05 | 73.2 | — | — | — | — | |
| MM1Vision Encoder=CLIP-L, Resolution=1792, LLM=7B, Data=1B2025.01 | 72.3 | — | — | — | — | |
| MM1-ChatSize=7B, # tokens per image=720, Zero-shot=true2024.05 | 72.3 | — | — | — | — | |
| MG-LLaVAVision Encoder=ConvNeXt-L, CLIP-L, RAMPlus, OWL-ViTv2-L, Resolution=768/336, LLM=Vicuna-7B, Data=2.5M2025.01 | 72.1 | — | — | — | — | |
| LLaVA-NeXTSize=13B, # tokens per image=2880, Zero-shot=true2024.05 | 70 | — | — | — | — | |
| Upper BoundBase Model=LLaVA-NeXT-13B2026.04 | 70 | — | — | — | — | |
| Mipha-3BSize=Size < 4B2024.02 | 69.7 | — | — | — | — | |
| Prompt Highlighter2023.12 | 69.5 | — | — | — | — | |
| SlimeVision Encoder=CLIP-L, Resolution=2016, LLM=Vicuna-7B, Data=2M2025.01 | 69.3 | — | — | — | — | |
| Mini-GeminiVision Encoder=ConvNeXt-L, CLIP-L, Resolution=768/336, LLM=Vicuna-7B, Data=2.8M2025.01 | 69.3 | — | — | — | — | |
| LLaVA-1.5 Vicuna-13B w/ EasyGenInstruction Tuning Data (IT)=665K2023.10 | 69.2 | — | — | — | — | |
| SVIT-v1.5-13BSize=> 8B2024.02 | 69.1 | — | — | — | — | |
| MiniCPM-V 2.0Size=Size < 4B2024.02 | 69.1 | — | — | — | — | |
| PIIP-LLaVAVision Encoder=ConvNeXt-B, CLIP-L, Resolution=1024/336, LLM=Vicuna-13B, Data=1.2M2025.01 | 68.5 | — | — | — | — | |
| LLaVA-v1.5-13B (LORA)Size=> 8B2024.02 | 68.45 | — | — | — | — | |
| Yi-VL-6BSize=4B < Size < 8B2024.02 | 68.4 | — | — | — | — | |
| LLaVA-UHD v2Vision Encoder=CLIP-L, Resolution=1008, LLM=Vicuna-7B, Data=1.4M2025.01 | 68.2 | — | — | — | — | |
| LLaVA-UHDVision Encoder=CLIP-L, Resolution=1008, LLM=Vicuna-13B, Data=1.2M2025.01 | 68 | — | — | — | — | |
| Claude 3 Sonnetsetup=zero-shot2024.05 | 67.8 | — | — | — | — | |
| LLaVA-1.5LLM=Vicuna-13B, Instruction Tuning Data (IT)=665K2023.10 | 67.8 | — | — | — | — | |
| LLaVA-1.5Vision Encoder=CLIP-L, Resolution=336, LLM=Vicuna-13B, Data=1.2M2025.01 | 67.7 | — | — | — | — | |
| PIIP-LLaVAVision Encoder=ConvNeXt-B, CLIP-L, Resolution=1024/336, LLM=Vicuna-7B, Data=1.2M2025.01 | 67.5 | — | — | — | — | |
| PIIP-LLaVAVision Encoder=ConvNeXt-L, CLIP-L, Resolution=1024/336, LLM=Vicuna-7B, Data=1.2M2025.01 | 67 | — | — | — | — | |
| LLaVA-v1.52023.12 | 67 | — | — | — | — | |
| LLaVA-HRVision Encoder=ConvNeXt-L, CLIP-L, Resolution=1024/448, LLM=Vicuna-13B, Data=1.2M2025.01 | 66.5 | — | — | — | — | |
| PIIP-LLaVAVision Encoder=ConvNeXt-L, CLIP-L, Resolution=1024/336, LLM=Vicuna-13B, Data=1.2M2025.01 | 66.5 | — | — | — | — | |
| LLaVA-HRVision Encoder=ConvNeXt-L, CLIP-L, Resolution=1024/448, LLM=Vicuna-7B, Data=1.2M2025.01 | 66.4 | — | — | — | — | |
| LLaVA-v1.5-7B (LORA)Size=4B < Size < 8B2024.02 | 66.19 | — | — | — | — | |
| LLaVA-1.5 Vicuna-7B w/ EasyGenInstruction Tuning Data (IT)=665K2023.10 | 66.1 | — | — | — | — | |
| mPLUG-Owl-22023.12 | 66 | — | — | — | — | |
| mPLUG-Owl2Size=4B < Size < 8B2024.02 | 66 | — | — | — | — | |
| LLaVA-1.5LLM=Vicuna-7B, Instruction Tuning Data (IT)=665K2023.10 | 65.2 | — | — | — | — | |
| mPLUG-Owl2Vision Encoder=CLIP-L, Resolution=448, LLM=Llama-2-7B, Data=400M2025.01 | 64.5 | — | — | — | — | |
| LLaVA-1.5Vision Encoder=CLIP-L, Resolution=336, LLM=Vicuna-7B, Data=1.2M2025.01 | 64.3 | — | — | — | — | |
| EasyGen Vicuna-7B w/ ViT-LInstruction Tuning Data (IT)=251K2023.10 | 63.9 | — | — | — | — | |
| Claude 3 Opussetup=zero-shot2024.05 | 63.3 | — | — | — | — | |
| CLASPBase Model=LLaVA-NeXT-13B, Token Retention Budget=640 Tokens2026.04 | 63.2 | — | — | — | — | |
| CLASPBase Model=LLaVA-NeXT-13B, Token Retention Budget=320 Tokens2026.04 | 62.8 | — | — | — | — | |
| SparseVLMBase Model=LLaVA-NeXT-13B, Token Retention Budget=640 Tokens2026.04 | 62 | — | — | — | — | |
| QWen-VL-Chat2023.12 | 61.8 | — | — | — | — | |
| QwenVL-ChatVision Encoder=ViT-G, Resolution=448, LLM=Qwen-7B, Data=1.4B2025.01 | 60.9 | — | — | — | — | |
| SparseVLMBase Model=LLaVA-NeXT-13B, Token Retention Budget=320 Tokens2026.04 | 60.8 | — | — | — | — | |
| Claude 3 Haikusetup=zero-shot2024.05 | 60.7 | — | — | — | — | |
| ShikraVision Encoder=CLIP-L, Resolution=224, LLM=Vicuna-13B, Data=6M2025.01 | 60.2 | — | — | — | — | |
| Shikra-7BSize=4B < Size < 8B2024.02 | 60.2 | — | — | — | — | |
| CLASPBase Model=LLaVA-NeXT-13B, Token Retention Budget=160 Tokens2026.04 | 59.8 | — | — | — | — | |
| SPHINX-intern2Vision Encoder=ConvNeXt-XXL, DINOv2-g, Resolution=448, LLM=InternLM2-7B, Data=16M2025.01 | 57.9 | — | — | — | — | |
| SparseVLMBase Model=LLaVA-NeXT-13B, Token Retention Budget=160 Tokens2026.04 | 56.5 | — | — | — | — | |
| InstructBLIPVision Encoder=ViT-g, Resolution=224, LLM=Vicuna-13B, Data=130M2025.01 | 39.8 | — | — | — | — | |
| InstructBLIPVision Encoder=ViT-g, Resolution=224, LLM=Vicuna-7B, Data=130M2025.01 | 38.3 | — | — | — | — | |
| InstructBLIPLLM=Vicuna-7B, Instruction Tuning Data (IT)=1.2M2023.10 | 33.9 | — | — | — | — | |
| InstructBLIP-Vicuna-7BSize=4B < Size < 8B2024.02 | 33.9 | — | — | — | — | |
| MiniGPT-4Instruction Tuning Data (IT)=5M2023.10 | 23 | — | — | — | — | |
| ASPOBase Model=InstructBLIP, Model Scale=13B, Optimization Protocol=ASPO2025.05 | — | 35.1 | — | 45.84 | — | |
| ASPOBase Model=LLaVA-v1.5, Model Scale=7B, Optimization Protocol=ASPO2025.05 | — | 67.7 | — | 65.16 | — | |
| ASPOBase Model=LLaVA-v1.5, Model Scale=13B, Optimization Protocol=ASPO2025.05 | — | 70.7 | — | 68.8 | — | |
| BAGEL2025.08 | — | — | 85 | — | — | |
| BAGELParams=14.0B2025.09 | — | — | 85 | — | — | |
| CuMoPT+IT=0.5M+0.6M, Res.=336, LLM=Mistral-7B2024.05 | — | 69.6 | — | — | — | |
| Dense ConnectorPT+IT=0.5M+0.6M, Res.=384, LLM=Phi2-2.7B2024.05 | — | 70.5 | — | — | — | |
| Dense ConnectorPT+IT=0.5M+0.6M, Res.=384, LLM=Vicuna-7B2024.05 | — | 68.4 | — | — | — | |
| Dense ConnectorPT+IT=0.5M+0.6M, Res.=384, LLM=Vicuna-13B2024.05 | — | 71.4 | — | — | — | |
| Dense ConnectorPT+IT=0.5M+0.6M, Res.=384, LLM=Llama3-8B2024.05 | — | 74.4 | — | — | — | |
| Dense ConnectorPT+IT=0.5M+0.6M, Res.=384, LLM=Yi-34B LORA2024.05 | — | 77.7 | — | — | — | |
| Dense ConnectorPT+IT=0.5M+0.6M, Res.=384, LLM=Llama3-70B LORA2024.05 | — | 79.4 | — | — | — | |
| Dense ConnectorPT+IT=1.2M+1.5M, Res.=384, LLM=Vicuna-13B2024.05 | — | 74.4 | — | — | — | |
| Dense ConnectorPT+IT=1.2M+1.5M, Res.=384 AnyRes, LLM=Vicuna-7B2024.05 | — | 69.2 | — | — | — | |
| Dense ConnectorPT+IT=1.2M+1.5M, Res.=384 AnyRes, LLM=Vicuna-13B2024.05 | — | 72.3 | — | — | — | |
| Dense ConnectorPT+IT=1.2M+1.5M, Res.=384 AnyRes, LLM=Yi-34B2024.05 | — | 81.2 | — | — | — | |
| DIM-4.6B-T2I/EditParams=3.0B | 1.6B2025.09 | — | — | 78.6 | — | — | |
| DPOBase Model=InstructBLIP, Model Scale=13B, Optimization Protocol=DPO2025.05 | — | 34.3 | — | 44.5 | — | |
| DPOBase Model=LLaVA-v1.5, Model Scale=7B, Optimization Protocol=DPO2025.05 | — | 66.4 | — | 63.12 | — | |
| DPOBase Model=LLaVA-v1.5, Model Scale=13B, Optimization Protocol=DPO2025.05 | — | 69.9 | — | 67.03 | — | |
| Emu3-GenParams=8.0B2025.09 | — | — | 58.5 | — | — | |
| FastVBase Model=LLaVA1.5-7B, Pruning Family=LLM Single-Layer Pruning2026.02 | — | 48 | — | — | 74.2 | |
| GPT-4V2025.08 | — | — | 74.3 | — | — | |
| HoneybeeLLM=Vicuna-7B, Projector=C-Abstractor, Vision Encoder=CLIP ViT-L/14, Res.=224, Visual tokens (M)=1442023.12 | — | — | 70.1 | — | — | |
| HoneybeeLLM=Vicuna-7B, Projector=D-Abstractor, Vision Encoder=CLIP ViT-L/14, Res.=224, Visual tokens (M)=1442023.12 | — | — | 70.8 | — | — | |
| HoneybeeLLM=Vicuna-13B, Projector=C-Abstractor, Vision Encoder=CLIP ViT-L/14, Res.=336, Visual tokens (M)=2562023.12 | — | — | 73.2 | — | — | |
| HoneybeeLLM=Vicuna-13B, Projector=D-Abstractor, Vision Encoder=CLIP ViT-L/14, Res.=336, Visual tokens (M)=2562023.12 | — | — | 73.5 | — | — | |
| IDEFICSLLM=LLaMA-7B, Projector=Flamingo, Vision Encoder=OpenCLIP ViT-H/14, Res.=2242023.12 | — | — | 48.2 | — | — | |
| InstructBLIPVision Encoder=ViT-g (1.3B), Language Model=Vicuna (7B), Zero-shot=true2023.11 | — | 36 | — | — | — |