List your company
ResearchFormerly WizWand

Multimodal Understanding on MMMU (Accuracy)

56.8AccuracyGPT-4V
21.12830.38939.6548.911Jan 13, 2025Mar 23, 2025May 31, 2025Aug 9, 2025Oct 17, 2025Dec 25, 2025Mar 5, 2026
Updated 7mo ago

Evaluation results

MethodLinks
GPT-4VLanguage Model Scale=Large, Training Data Scale=Large2025.0156.8
Llama 3.2-11BLanguage Model Scale=11B, Training Data Scale=Large2025.0150.7
Cambrian-1-34BLanguage Model Scale=34B, Training Data Scale=Large2025.0149.7
MASQuantModel=Qwen2.5-VL-3B, Bits=W4A82026.0346.7
DenseModel=Qwen2.5-VL-7B, Bits=FP162026.0346.7
MBQModel=Qwen2.5-VL-7B, Bits=W8A82026.0346.7
MASQuantModel=Qwen2.5-VL-3B, Bits=W8A82026.0346.6
MASQuantModel=Qwen2.5-VL-7B, Bits=W8A82026.0346.2
RTNModel=Qwen2.5-VL-7B, Bits=W8A82026.0345.6
MBQModel=Qwen2.5-VL-7B, Bits=W4A162026.0344.4
MASQuantModel=Qwen2.5-VL-7B, Bits=W4A162026.0344.4
MASQuantModel=Qwen2.5-VL-3B, Bits=W4A162026.0343.3
MBQModel=Qwen2.5-VL-3B, Bits=W8A82026.0343.3
RTNModel=Qwen2.5-VL-7B, Bits=W4A162026.0343.3
AWQModel=Qwen2.5-VL-7B, Bits=W4A162026.0343.3
SQModel=Qwen2.5-VL-7B, Bits=W8A82026.0343.3
RTNModel=Qwen2.5-VL-7B, Bits=W4A82026.0343.3
MBQModel=Qwen2.5-VL-7B, Bits=W4A82026.0343.3
MASQuantModel=Qwen2.5-VL-7B, Bits=W4A82026.0343.3
RTNModel=Qwen2.5-VL-3B, Bits=W8A82026.0342.6
SQModel=Qwen2.5-VL-3B, Bits=W8A82026.0342.6
DenseModel=Qwen2.5-VL-3B, Bits=FP162026.0342.2
AWQModel=Qwen2.5-VL-3B, Bits=W4A162026.0341.9
MBQModel=Qwen2.5-VL-3B, Bits=W4A162026.0341.9
MBQModel=Qwen2.5-VL-3B, Bits=W4A82026.0341.2
RTNModel=Qwen2.5-VL-3B, Bits=W4A162026.0340.9
MASQuantModel=Qwen2.5-VL-3B, Bits=W4A62026.0340
MBQModel=Qwen2.5-VL-3B, Bits=W4A62026.0338.7
VILA-1.5Language Model Scale=7B, Training Data Scale=Standard2025.0138.6
SQModel=Qwen2.5-VL-7B, Bits=W4A82026.0337.8
LeoLanguage Model Scale=7B, Training Data Scale=Standard2025.0136.4
LLaVA-NeXtLanguage Model Scale=7B, Training Data Scale=Standard2025.0135.8
MBQModel=Qwen2.5-VL-7B, Bits=W4A62026.0330
MASQuantModel=Qwen2.5-VL-7B, Bits=W4A62026.0329.7
SQModel=Qwen2.5-VL-7B, Bits=W4A62026.0328.9
RTNModel=Qwen2.5-VL-3B, Bits=W4A82026.0325.6
SQModel=Qwen2.5-VL-3B, Bits=W4A82026.0325.6
SQModel=Qwen2.5-VL-3B, Bits=W4A62026.0322.5