Vision-Language Understanding on V&L Benchmark Suite (MMB, MMStar, MMMU, etc.) (test val)
80.7MMB ScoreQwen2-VL-7B (Teacher)
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Qwen2-VL-7B (Teacher)Bitwidth=BF162026.01 | 80.7 | 60.7 | 54.1 | 50.6 | 83 | 84.5 | 76.9 | 83.3 | 71.7 | |
| GRACEBitwidth=BF162026.01 | 77.9 | 55.5 | 52 | 48.3 | 80 | 83.1 | 75.7 | 81 | 69.2 | |
| GRACEBitwidth=8-bit2026.01 | 77.4 | 55.2 | 51.8 | 47.9 | 79.7 | 82.9 | 75.5 | 80.8 | 68.9 | |
| GRACEBitwidth=4-bit2026.01 | 76.9 | 54.3 | 51.1 | 46.5 | 78.6 | 81.4 | 75.6 | 79.1 | 68 | |
| Qwen2-VL-2B (Baseline)Bitwidth=BF162026.01 | 72.6 | 50.4 | 45.4 | 42.9 | 73.1 | 81 | 72.7 | 73.7 | 64 | |
| RTNBitwidth=4-bit2026.01 | 70.98 | 45.07 | 37.22 | 42.74 | 71.15 | 78.8 | 72.45 | 70.67 | 61.1 | |
| MBQBitwidth=4-bit2026.01 | 70.55 | 44.53 | 38.22 | 39.89 | 70.21 | 80.9 | 71.86 | 71.72 | 61 | |
| GPTQBitwidth=4-bit2026.01 | 70.51 | 46.33 | 36.22 | 39.62 | 70.53 | 79.5 | 72.62 | 72.1 | 60.9 | |
| SPEED-QBitwidth=4-bit2026.01 | 69.85 | 50.87 | 42 | 41.71 | 70.92 | 76.5 | 74.4 | 72.01 | 62.3 | |
| AWQBitwidth=4-bit2026.01 | 68.89 | 44.8 | 37.33 | 39.55 | 70.08 | 78.9 | 71.83 | 72.15 | 60.4 |