Multimodal Understanding on MMMU (Accuracy)
56.8AccuracyGPT-4V
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-4VLanguage Model Scale=Large, Training Data Scale=Large2025.01 | 56.8 | |
| Llama 3.2-11BLanguage Model Scale=11B, Training Data Scale=Large2025.01 | 50.7 | |
| Cambrian-1-34BLanguage Model Scale=34B, Training Data Scale=Large2025.01 | 49.7 | |
| MASQuantModel=Qwen2.5-VL-3B, Bits=W4A82026.03 | 46.7 | |
| DenseModel=Qwen2.5-VL-7B, Bits=FP162026.03 | 46.7 | |
| MBQModel=Qwen2.5-VL-7B, Bits=W8A82026.03 | 46.7 | |
| MASQuantModel=Qwen2.5-VL-3B, Bits=W8A82026.03 | 46.6 | |
| MASQuantModel=Qwen2.5-VL-7B, Bits=W8A82026.03 | 46.2 | |
| RTNModel=Qwen2.5-VL-7B, Bits=W8A82026.03 | 45.6 | |
| MBQModel=Qwen2.5-VL-7B, Bits=W4A162026.03 | 44.4 | |
| MASQuantModel=Qwen2.5-VL-7B, Bits=W4A162026.03 | 44.4 | |
| MASQuantModel=Qwen2.5-VL-3B, Bits=W4A162026.03 | 43.3 | |
| MBQModel=Qwen2.5-VL-3B, Bits=W8A82026.03 | 43.3 | |
| RTNModel=Qwen2.5-VL-7B, Bits=W4A162026.03 | 43.3 | |
| AWQModel=Qwen2.5-VL-7B, Bits=W4A162026.03 | 43.3 | |
| SQModel=Qwen2.5-VL-7B, Bits=W8A82026.03 | 43.3 | |
| RTNModel=Qwen2.5-VL-7B, Bits=W4A82026.03 | 43.3 | |
| MBQModel=Qwen2.5-VL-7B, Bits=W4A82026.03 | 43.3 | |
| MASQuantModel=Qwen2.5-VL-7B, Bits=W4A82026.03 | 43.3 | |
| RTNModel=Qwen2.5-VL-3B, Bits=W8A82026.03 | 42.6 | |
| SQModel=Qwen2.5-VL-3B, Bits=W8A82026.03 | 42.6 | |
| DenseModel=Qwen2.5-VL-3B, Bits=FP162026.03 | 42.2 | |
| AWQModel=Qwen2.5-VL-3B, Bits=W4A162026.03 | 41.9 | |
| MBQModel=Qwen2.5-VL-3B, Bits=W4A162026.03 | 41.9 | |
| MBQModel=Qwen2.5-VL-3B, Bits=W4A82026.03 | 41.2 | |
| RTNModel=Qwen2.5-VL-3B, Bits=W4A162026.03 | 40.9 | |
| MASQuantModel=Qwen2.5-VL-3B, Bits=W4A62026.03 | 40 | |
| MBQModel=Qwen2.5-VL-3B, Bits=W4A62026.03 | 38.7 | |
| VILA-1.5Language Model Scale=7B, Training Data Scale=Standard2025.01 | 38.6 | |
| SQModel=Qwen2.5-VL-7B, Bits=W4A82026.03 | 37.8 | |
| LeoLanguage Model Scale=7B, Training Data Scale=Standard2025.01 | 36.4 | |
| LLaVA-NeXtLanguage Model Scale=7B, Training Data Scale=Standard2025.01 | 35.8 | |
| MBQModel=Qwen2.5-VL-7B, Bits=W4A62026.03 | 30 | |
| MASQuantModel=Qwen2.5-VL-7B, Bits=W4A62026.03 | 29.7 | |
| SQModel=Qwen2.5-VL-7B, Bits=W4A62026.03 | 28.9 | |
| RTNModel=Qwen2.5-VL-3B, Bits=W4A82026.03 | 25.6 | |
| SQModel=Qwen2.5-VL-3B, Bits=W4A82026.03 | 25.6 | |
| SQModel=Qwen2.5-VL-3B, Bits=W4A62026.03 | 22.5 |