List your company
ResearchFormerly WizWand

Multimodal Understanding on MMBench v1.1 (test)

85.8ScoreQwen-VL-Max-0809
46.17656.46366.7577.037Oct 17, 2024
Updated 8mo ago

Evaluation results

MethodLinks
Qwen-VL-Max-0809Params (B)=722024.1085.8—
InternVL2-Llama3-76BParams (B)=762024.1085.5—
GPT-4o-202408062024.1080.5—
Claude3.5-Sonnet2024.1078.5—
Gemini-1.5-Pro2024.1073.9—
Qwen2-VL-2BParams (B)=2.12024.1072.2—
Gemini-1.0-Pro2024.1069.7—
InternVL2-2BParams (B)=2.12024.1069.6—
MiniMonkeyParams (B)=2.22024.1068.9—
Qwen-VL-Plus2024.1066.2—
MiniCPM-V-2Params (B)=2.82024.1065.8—
PaliGemma-3B-mix-448Params (B)=2.92024.1065.6—
GPT-4v (1106, detail-high)2024.1065.5—
Phi-3-VisionParams (B)=4.22024.1065.2—
H2OVL-Mississippi-2BParams (B)=2.12024.1064.8—
Claude3-Sonnet2024.1063.9—
DeepSeek-VL-1.3BParams (B)=2.02024.1063.8—
InternVL2-1BParams (B)=0.82024.1059.7—
H2OVL-Mississippi-0.8BParams (B)=0.82024.1047.7—
AgoraRouting=Pool of 5 baseline VLMs2026.01—89.5
FastVBase Model=Qwen2-VL-7B, Token Budget=~1/32026.01—77.8
FastVBase Model=Qwen2.5-VL-7B, Token Budget=~1/32026.01—77.5
gemini-2.0-flash2026.01—83
gemini-2.5-pro-exp-03-252026.01—88.3
gemma-3-27bScale=27B2026.01—78.9
gpt-4o-2024-08-062026.01—74.3
gpt-4o-mini2026.01—76.3
InternVL3-78BScale=78B2026.01—87.7
PDropBase Model=Qwen2-VL-7B, Token Budget=~1/32026.01—80.6
PDropBase Model=Qwen2.5-VL-7B, Token Budget=~1/32026.01—78.2
Qwen2-VL-7BBase Model=Qwen2-VL-7B, Token Budget=Full2026.01—80.7
Qwen2.5-VL-7BBase Model=Qwen2.5-VL-7B, Token Budget=Full2026.01—83.5
qwen2.5vl-72b-instructScale=72B2026.01—88.4
qwen2.5vl-7b-instructScale=7B2026.01—82.6
SparseVLMBase Model=Qwen2-VL-7B, Token Budget=~1/32026.01—79
SparseVLMBase Model=Qwen2.5-VL-7B, Token Budget=~1/32026.01—79.6
VisionTrimBase Model=Qwen2-VL-7B, Token Budget=~1/32026.01—82.8
VisionTrimBase Model=Qwen2.5-VL-7B, Token Budget=~1/32026.01—83.2