Multimodal Understanding and Reasoning on MMMU, SEED, OCRBench, VizWiz, ScienceQA, and TextVQA (test/val)
61.9MMMU ScoreAWQ
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| AWQModel=Qwen2-VL-72B, Bitwidth=W4A162024.12 | 61.9 | 77.5 | 79.5 | 75.8 | 92.2 | 82.4 | 78.2 | |
| FP16Model=Qwen2-VL-72B, Bitwidth=FP162024.12 | 61.1 | 77.6 | 79.9 | 76 | 91.6 | 82.5 | 78.1 | |
| MBQModel=Qwen2-VL-72B, Bitwidth=W4A162024.12 | 60.8 | 77.6 | 79.9 | 75.8 | 92.8 | 82.3 | 78.2 | |
| MBQModel=Qwen2-VL-72B, Bitwidth=W8A82024.12 | 60.7 | 77.6 | 79.9 | 75.9 | 91.4 | 82.4 | 78 | |
| RTNModel=Qwen2-VL-72B, Bitwidth=W8A82024.12 | 60.4 | 77.5 | 79.5 | 75.7 | 90.9 | 82.2 | 77.7 | |
| SQModel=Qwen2-VL-72B, Bitwidth=W8A82024.12 | 60.3 | 77.5 | 79.7 | 76.1 | 91.3 | 82 | 77.8 | |
| RTNModel=Qwen2-VL-72B, Bitwidth=W4A162024.12 | 59.8 | 77.7 | 79.6 | 75.8 | 91.3 | 82.6 | 77.8 | |
| RTNModel=LLaVA-onevision-72B, Bitwidth=W8A82024.12 | 56.8 | 78 | 73.1 | 69.4 | 90.3 | 79.2 | 74.5 | |
| MBQModel=LLaVA-onevision-72B, Bitwidth=W4A162024.12 | 56.4 | 77.9 | 72.3 | 69 | 90.3 | 79.3 | 74.2 | |
| SQModel=LLaVA-onevision-72B, Bitwidth=W8A82024.12 | 56.3 | 78 | 72.7 | 69.2 | 89.7 | 79 | 74.2 | |
| RTNModel=LLaVA-onevision-72B, Bitwidth=W4A162024.12 | 56.2 | 77.9 | 72.1 | 68.8 | 90.4 | 78.9 | 74.1 | |
| MBQModel=LLaVA-onevision-72B, Bitwidth=W8A82024.12 | 56.2 | 78.1 | 73.1 | 69.2 | 89.8 | 79.1 | 74.3 | |
| FP16Model=LLaVA-onevision-72B, Bitwidth=FP162024.12 | 56.1 | 78.1 | 73.2 | 69.2 | 90 | 79.3 | 74.3 | |
| FP16Model=Qwen2-VL-7B, Bitwidth=FP162024.12 | 50.6 | 76.4 | 80.7 | 68.3 | 85.1 | 82 | 73.8 | |
| RTNModel=Qwen2-VL-7B, Bitwidth=W4A162024.12 | 50.2 | 76 | 80.1 | 67.4 | 84.5 | 81.2 | 73.2 | |
| AWQModel=Qwen2-VL-7B, Bitwidth=W4A162024.12 | 50.1 | 76.1 | 80.4 | 68.4 | 85 | 81.7 | 73.6 | |
| SQModel=Qwen2-VL-7B, Bitwidth=W8A82024.12 | 50.1 | 76.3 | 80.6 | 68.5 | 85 | 81.5 | 73.7 | |
| MBQModel=Qwen2-VL-7B, Bitwidth=W8A82024.12 | 50.1 | 76.4 | 80.7 | 68.3 | 85.4 | 81.8 | 73.8 | |
| MBQModel=Qwen2-VL-7B, Bitwidth=W4A162024.12 | 50 | 76.3 | 80.8 | 68.6 | 84.6 | 81.4 | 73.6 | |
| RTNModel=Qwen2-VL-7B, Bitwidth=W8A82024.12 | 49.4 | 76.3 | 80.9 | 68.2 | 84.5 | 81.2 | 73.4 | |
| MBQModel=InternVL2-8B, Bitwidth=W4A162024.12 | 48.9 | 75.9 | 76.7 | 60.8 | 96.3 | 76.5 | 72.5 | |
| RTNModel=InternVL2-26B, Bitwidth=W4A162024.12 | 48.2 | 76.8 | 78 | 64.6 | 97.1 | 81.8 | 74.4 | |
| SQModel=InternVL2-26B, Bitwidth=W8A82024.12 | 48.1 | 76.7 | 78.3 | 65.5 | 97.4 | 82 | 74.7 | |
| FP16Model=InternVL2-8B, Bitwidth=FP162024.12 | 48 | 76 | 76.5 | 61.1 | 96.2 | 77 | 72.5 | |
| SQModel=InternVL2-8B, Bitwidth=W8A82024.12 | 48 | 76.1 | 77.1 | 61 | 96.1 | 76.9 | 72.5 | |
| MBQModel=InternVL2-8B, Bitwidth=W8A82024.12 | 48 | 76 | 77.4 | 61 | 96.4 | 77 | 72.6 | |
| MBQModel=InternVL2-26B, Bitwidth=W8A82024.12 | 47.9 | 76.8 | 78.1 | 66.2 | 97.5 | 82 | 74.8 | |
| RTNModel=InternVL2-8B, Bitwidth=W4A162024.12 | 47.6 | 75.9 | 75.6 | 60.1 | 96 | 76.2 | 71.9 | |
| RTNModel=InternVL2-8B, Bitwidth=W8A82024.12 | 47.4 | 76.2 | 77.3 | 61 | 96.2 | 76.9 | 72.5 | |
| AWQModel=InternVL2-26B, Bitwidth=W4A162024.12 | 47.4 | 76.8 | 77.1 | 65.9 | 97.3 | 82 | 74.4 | |
| RTNModel=InternVL2-26B, Bitwidth=W8A82024.12 | 47.4 | 76.5 | 78.4 | 65.1 | 97.3 | 81.7 | 74.4 | |
| MBQModel=InternVL2-26B, Bitwidth=W4A162024.12 | 47.2 | 76.8 | 77.5 | 65.4 | 97.5 | 82.1 | 74.4 | |
| AWQModel=InternVL2-8B, Bitwidth=W4A162024.12 | 47.1 | 75.8 | 76.7 | 60.1 | 96.3 | 76.4 | 72.1 | |
| FP16Model=InternVL2-26B, Bitwidth=FP162024.12 | 47.1 | 76.8 | 77.9 | 66.2 | 97.5 | 82.1 | 74.6 | |
| RTNModel=LLaVA-onevision-7B, Bitwidth=W8A82024.12 | 46.3 | 74.8 | 63.5 | 60.5 | 90.3 | 75.9 | 68.6 | |
| FP16Model=LLaVA-onevision-7B, Bitwidth=FP162024.12 | 46 | 74.9 | 62.2 | 60.4 | 85.4 | 76.1 | 67.5 | |
| SQModel=LLaVA-onevision-7B, Bitwidth=W8A82024.12 | 46 | 74.9 | 63.2 | 60.7 | 90.3 | 75.7 | 68.5 | |
| MBQModel=LLaVA-onevision-7B, Bitwidth=W8A82024.12 | 45.6 | 74.7 | 62.6 | 61 | 90.2 | 75.7 | 68.3 | |
| RTNModel=LLaVA-onevision-7B, Bitwidth=W4A162024.12 | 44.9 | 74.6 | 61.7 | 59.6 | 89.8 | 75.3 | 67.6 | |
| AWQModel=LLaVA-onevision-7B, Bitwidth=W4A162024.12 | 44.6 | 74.7 | 61.8 | 59.1 | 90.1 | 75.8 | 67.7 | |
| MBQModel=LLaVA-onevision-7B, Bitwidth=W4A162024.12 | 44.4 | 74.7 | 62.1 | 59.3 | 90.2 | 75.6 | 67.7 | |
| AWQModel=LLaVA-onevision-72B, Bitwidth=W4A162024.12 | 39.1 | 75.9 | 58.1 | 59.9 | 80.4 | 69.1 | 63.8 |