Vision Understanding on MMMU
72.9AccuracyGPT-4o
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| GPT-4o2025.06 | 72.9 | — | — | |
| LLaVA-1.52025.06 | 67.8 | — | — | |
| MetaQuery2025.06 | 58.6 | — | — | |
| UniWorld-V12025.06 | 58.6 | — | — | |
| BAGEL2025.06 | 55.3 | — | — | |
| BLIP3-o2025.06 | 50.6 | — | — | |
| DenseModel=Qwen2.5-Omni-7B, Bits=FP162026.03 | 50 | — | — | |
| MBQModel=Qwen2.5-Omni-7B, Bits=W8A82026.03 | 50 | — | — | |
| MASQuantModel=Qwen2.5-Omni-7B, Bits=W8A82026.03 | 49.9 | — | — | |
| RTNModel=Qwen2.5-Omni-7B, Bits=W8A82026.03 | 48.9 | — | — | |
| MASQuantModel=Qwen2.5-Omni-7B, Bits=W4A162026.03 | 48.8 | — | — | |
| SQModel=Qwen2.5-Omni-7B, Bits=W8A82026.03 | 48.8 | — | — | |
| MASQuantModel=Qwen2.5-Omni-7B, Bits=W4A82026.03 | 48.8 | — | — | |
| AWQModel=Qwen2.5-Omni-7B, Bits=W4A162026.03 | 47.8 | — | — | |
| MBQModel=Qwen2.5-Omni-7B, Bits=W4A162026.03 | 47.8 | — | — | |
| MBQModel=Qwen2.5-Omni-7B, Bits=W4A82026.03 | 47.8 | — | — | |
| DAVELLM Backbone=Qwen-2.5-7B-Instruct2025.12 | 45.8 | — | — | |
| Qwen2.5-Omni (Ours)W/A Bits=4/42026.06 | 45.11 | — | — | |
| LaViDa-O2025.12 | 45.1 | — | — | |
| FP16 (Full Precision)W/A Bits=16/162026.06 | 44.78 | — | — | |
| RTNModel=Qwen2.5-Omni-7B, Bits=W4A162026.03 | 44.4 | — | — | |
| SigLIP 2LLM Backbone=Qwen-2.5-7B-Instruct2025.12 | 43.9 | — | — | |
| Sparse-LaViDa2025.12 | 43.6 | — | — | |
| QLoRAW/A Bits=4/162026.06 | 43.56 | — | — | |
| DenseModel=Qwen2.5-Omni-3B, Bits=FP162026.03 | 43.3 | — | — | |
| MASQuantModel=Qwen2.5-Omni-3B, Bits=W8A82026.03 | 42.3 | — | — | |
| RTNModel=Qwen2.5-Omni-7B, Bits=W4A82026.03 | 42.2 | — | — | |
| SQModel=Qwen2.5-Omni-7B, Bits=W4A82026.03 | 42.2 | — | — | |
| PoMQ-ViTW/A Bits=4/42026.06 | 42.11 | — | — | |
| MBQModel=Qwen2.5-Omni-3B, Bits=W8A82026.03 | 41.8 | — | — | |
| SQModel=Qwen2.5-Omni-3B, Bits=W8A82026.03 | 41.7 | — | — | |
| AIMv2LLM Backbone=Qwen-2.5-7B-Instruct2025.12 | 41.3 | — | — | |
| RTNModel=Qwen2.5-Omni-3B, Bits=W8A82026.03 | 41.3 | — | — | |
| QVLMW/A Bits=4/42026.06 | 40.33 | — | — | |
| DophineLLM Backbone=Qwen-2.5-7B-Instruct2025.12 | 40.2 | — | — | |
| Naive W4A4W/A Bits=4/42026.06 | 40 | — | — | |
| MASQuantModel=Qwen2.5-Omni-3B, Bits=W4A162026.03 | 39.6 | — | — | |
| MBQModel=Qwen2.5-Omni-3B, Bits=W4A162026.03 | 38.9 | — | — | |
| MAE-WebLLM Backbone=Qwen-2.5-7B-Instruct2025.12 | 38.4 | — | — | |
| Pix2structLLM Backbone=Qwen-2.5-7B-Instruct2025.12 | 37.9 | — | — | |
| SQModel=Qwen2.5-Omni-7B, Bits=W4A62026.03 | 37.8 | — | — | |
| SigLIP 2LLM Backbone=Llama-3.2-3B-Instruct2025.12 | 36.9 | — | — | |
| MASQuantModel=Qwen2.5-Omni-7B, Bits=W4A62026.03 | 36.8 | — | — | |
| AWQModel=Qwen2.5-Omni-3B, Bits=W4A162026.03 | 36.7 | — | — | |
| MASQuantModel=Qwen2.5-Omni-3B, Bits=W4A82026.03 | 36.7 | — | — | |
| DAVELLM Backbone=Llama-3.2-3B-Instruct2025.12 | 36.6 | — | — | |
| Janus-Pro2025.06 | 36.3 | — | — | |
| AIMv2LLM Backbone=Llama-3.2-3B-Instruct2025.12 | 36.3 | — | — | |
| SeGroSFine-Tuning=Ours2026.03 | 36 | — | — | |
| SFTFine-Tuning=SFT2026.03 | 35.3 | — | — | |
| DinoV2LLM Backbone=Llama-3.2-3B-Instruct2025.12 | 35 | — | — | |
| RecaFine-Tuning=Reca2026.03 | 34.9 | — | — | |
| Base ModelFine-Tuning=w/o SFT2026.03 | 34.7 | — | — | |
| Pix2StructLLM Backbone=Llama-3.2-3B-Instruct2025.12 | 33.3 | — | — | |
| DophineLLM Backbone=Llama-3.2-3B-Instruct2025.12 | 33.3 | — | — | |
| MBQModel=Qwen2.5-Omni-7B, Bits=W4A62026.03 | 33.3 | — | — | |
| Video-LLaVA2025.06 | 32.8 | — | — | |
| RTNModel=Qwen2.5-Omni-3B, Bits=W4A162026.03 | 32.2 | — | — | |
| MASQuantModel=Qwen2.5-Omni-3B, Bits=W4A62026.03 | 32.2 | — | — | |
| Emu32025.06 | 31.6 | — | — | |
| MBQModel=Qwen2.5-Omni-3B, Bits=W4A62026.03 | 31.4 | — | — | |
| Janus2025.06 | 30.5 | — | — | |
| SQModel=Qwen2.5-Omni-3B, Bits=W4A82026.03 | 30 | — | — | |
| MAE-WebLLM Backbone=Llama-3.2-3B-Instruct2025.12 | 29.9 | — | — | |
| RTNModel=Qwen2.5-Omni-3B, Bits=W4A82026.03 | 28.9 | — | — | |
| RTNModel=Qwen2.5-Omni-3B, Bits=W4A62026.03 | 28.9 | — | — | |
| SQModel=Qwen2.5-Omni-3B, Bits=W4A62026.03 | 28.9 | — | — | |
| RTNModel=Qwen2.5-Omni-7B, Bits=W4A62026.03 | 28.9 | — | — | |
| DiTLLM Backbone=Llama-3.2-3B-Instruct2025.12 | 28.8 | — | — | |
| MBQModel=Qwen2.5-Omni-3B, Bits=W4A82026.03 | 27.8 | — | — | |
| Show-o2025.06 | 27.4 | — | — | |
| BagelType=Und. and Gen., Params=7B2026.01 | — | 55.3 | — | |
| BAGELType=Unified, # LLM Params=1.5B2026.03 | — | 43.2 | — | |
| BAGELType=Unified (>1.5B), # LLM Params=7B2026.03 | — | 55.3 | — | |
| BAGELModel Category=Unified Understanding & Generation Model2025.10 | — | — | 55.3 | |
| BAGEL-1.5BStage=Uni. 1 Stage2025.12 | — | — | 43.2 | |
| BAGEL-7BStage=Uni. 1 Stage2025.12 | — | — | 55.3 | |
| BAGEL-7BLLM Params=3B, Model Category=Und. and Gen.2026.06 | — | — | 43.2 | |
| BLIP-3Type=Und. Only, # LLM Params=4B2026.03 | — | 41.1 | — | |
| BLIP3-o-4BLLM Params=4B, Model Category=Und. and Gen.2026.06 | — | — | 46.6 | |
| ChameleonType=Unified (>1.5B), # LLM Params=7B2026.03 | — | 28.4 | — | |
| Chameleon-7bType=Unified (Understanding & Generation), LLM Params=7B2025.03 | — | 22.4 | — | |
| Chameleon-7BLLM Params=7B, Model Category=Und. and Gen.2026.06 | — | — | 28.4 | |
| DeepSeek-VL2Type=Und. Only, # LLM Params=4.1B/28B2026.03 | — | 51.1 | — | |
| Emu2-ChatType=Understanding-only, LLM Params=33B2025.03 | — | 40.7 | — | |
| Emu3Model Category=Unified Understanding & Generation Model2025.10 | — | — | 31.6 | |
| Emu3-ChatSize=8.0B, Capability=Und. only2025.05 | — | 31.6 | — | |
| Emu3-ChatType=Und. Only, # LLM Params=8B2026.03 | — | 31.6 | — | |
| Emu3-Chat-8BLLM Params=8B, Model Category=Und. Only2026.06 | — | — | 31.6 | |
| Gemini-Nano-1Type=Unified (Understanding & Generation), LLM Params=1.8B2025.03 | — | 26.3 | — | |
| Harmon-1.5BLLM Params=1.5B, Model Category=Und. and Gen.2026.06 | — | — | 38.9 | |
| IdeficsType=Understanding-only, LLM Params=9B2025.03 | — | 18.4 | — | |
| ILLUMEType=Unified (>1.5B), # LLM Params=7B2026.03 | — | 38.2 | — | |
| InternVL2Type=Und. Only, # LLM Params=1.8B2026.03 | — | 34.3 | — | |
| InternVL2Type=Und. Only, # LLM Params=7B2026.03 | — | 49.3 | — | |
| InternVL2.5Type=Und. Only, # LLM Params=1.8B2026.03 | — | 43.6 | — | |
| InternVL2.5Type=Und. Only, # LLM Params=7B2026.03 | — | 56 | — | |
| InternVL3-1BLLM Params=1B, Model Category=Und. Only2026.06 | — | — | 43.4 | |
| InternVL3-2BLLM Params=2B, Model Category=Und. Only2026.06 | — | — | 48.2 | |
| InternVL3.5Type=Und. only, Params=4B2026.01 | — | 66.6 | — |