Multimodal Understanding on MMMU 1.0 (test)
50.6MMMU ScoreFP16 Baseline
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FP16 BaselineModel=Qwen2-VL-7B, Bitwidth=FP162024.12 | 50.6 | 73.8 | |
| FP16 BaselineModel=InternVL2-8B, Bitwidth=FP162024.12 | 48 | 72.5 | |
| MBQModel=Qwen2-VL-7B, Bitwidth=W3A162024.12 | 47.9 | 71.6 | |
| MBQModel=Qwen2-VL-7B, Bitwidth=W4A82024.12 | 47.2 | 68.4 | |
| MBQModel=InternVL2-8B, Bitwidth=W3A162024.12 | 46.9 | 71.1 | |
| FP16 BaselineModel=LLaVA-onevision-7B, Bitwidth=FP162024.12 | 46 | 67.5 | |
| MBQModel=InternVL2-8B, Bitwidth=W4A82024.12 | 45.6 | 69.6 | |
| MBQModel=LLaVA-onevision-7B, Bitwidth=W4A82024.12 | 42.6 | 63.1 | |
| MBQModel=LLaVA-onevision-7B, Bitwidth=W3A162024.12 | 42 | 65.3 | |
| GPTQModel=LLaVA-onevision-7B, Bitwidth=W3A162024.12 | 41.9 | 64.1 | |
| RTNModel=LLaVA-onevision-7B, Bitwidth=W4A82024.12 | 38.2 | 56.1 | |
| AWQModel=LLaVA-onevision-7B, Bitwidth=W3A162024.12 | 36.6 | 60.6 | |
| RTNModel=LLaVA-onevision-7B, Bitwidth=W3A162024.12 | 34.7 | 47.1 | |
| SQModel=LLaVA-onevision-7B, Bitwidth=W4A82024.12 | 30.9 | 51.1 |