Multimodal Evaluation on MME English
60.21ScoreFull Prec
Evaluation Results
| Method | Links | |
|---|---|---|
| Full PrecModel=Qwen2.5-VL-32B-Instruct, Bit-width=Full Precision2025.08 | 60.21 | |
| Full PrecModel=Qwen2.5-VL-7B-Instruct, Bit-width=Full Precision2025.08 | 58.55 | |
| Full PrecModel=LLaVA-OneVision-7B, Bit-width=Full Precision2025.08 | 57.39 | |
| Full PrecModel=Qwen2-VL-7B-Instruct, Bit-width=Full Precision2025.08 | 57.3 | |
| VLMQModel=Qwen2-VL-7B-Instruct, Quantization Bit=INT42025.08 | 55.88 | |
| GPTAQModel=Qwen2-VL-7B-Instruct, Quantization Bit=INT42025.08 | 55.55 | |
| GPTQModel=Qwen2-VL-7B-Instruct, Quantization Bit=INT42025.08 | 55.44 | |
| VLMQModel=Qwen2.5-VL-32B-Instruct, Bit-width=3-bit, Importance Factor Configuration=Default (ℓ1-norm)2025.08 | 53.05 | |
| GPTAQModel=Qwen2.5-VL-32B-Instruct, Bit-width=3-bit2025.08 | 52.13 | |
| GPTQModel=Qwen2.5-VL-32B-Instruct, Bit-width=3-bit2025.08 | 51.77 | |
| VLMQModel=Qwen2-VL-7B-Instruct, Bit-width=3-bit, Importance Factor Configuration=Default (ℓ1-norm)2025.08 | 50.17 | |
| GPTAQModel=Qwen2-VL-7B-Instruct, Bit-width=3-bit2025.08 | 49.76 | |
| VLMQModel=LLaVA-OneVision-7B, Bit-width=3-bit, Importance Factor Configuration=token-level ℓ2-norm2025.08 | 49.12 | |
| VLMQModel=LLaVA-OneVision-7B, Bit-width=3-bit, Importance Factor Configuration=Default (ℓ1-norm)2025.08 | 48.93 | |
| GPTQModel=Qwen2-VL-7B-Instruct, Bit-width=3-bit2025.08 | 47.76 | |
| GPTAQModel=Qwen2.5-VL-7B-Instruct, Bit-width=3-bit2025.08 | 47.18 | |
| GPTAQModel=LLaVA-OneVision-7B, Bit-width=3-bit2025.08 | 46.48 | |
| GPTQModel=LLaVA-OneVision-7B, Bit-width=3-bit2025.08 | 46.47 | |
| VLMQModel=Qwen2.5-VL-7B-Instruct, Bit-width=3-bit, Importance Factor Configuration=Default (ℓ1-norm)2025.08 | 45.5 | |
| GPTQBase Model=Qwen2.5-VL-7B-Instruct, Bit-width=2-bit, Group Size=1282025.08 | 44.94 | |
| GPTAQBase Model=LLaVA-OneVision-7B, Bit-width=2-bit, Group Size=1282025.08 | 42.59 | |
| GPTQModel=Qwen2.5-VL-7B-Instruct, Bit-width=3-bit2025.08 | 42.31 | |
| VLMQBase Model=Qwen2-VL-7B-Instruct, Bit-width=2-bit, Group Size=128, Precursor Algorithm=GPTQ2025.08 | 41.97 | |
| GPTQBase Model=Qwen2-VL-7B-Instruct, Bit-width=2-bit, Group Size=1282025.08 | 41.33 | |
| VLMQBase Model=Qwen2.5-VL-7B-Instruct, Bit-width=2-bit, Group Size=128, Precursor Algorithm=GPTQ2025.08 | 41.27 | |
| GPTQBase Model=LLaVA-OneVision-7B, Bit-width=2-bit, Group Size=1282025.08 | 40.29 | |
| Full PrecModel=Qwen2-VL-2B-Instruct, Bit-width=Full Precision2025.08 | 40.27 | |
| GPTAQBase Model=Qwen2.5-VL-7B-Instruct, Bit-width=2-bit, Group Size=1282025.08 | 39.38 | |
| Full PrecModel=LLaVA-OneVision-0.5B, Bit-width=Full Precision2025.08 | 38.94 | |
| GPTAQModel=Qwen2-VL-2B-Instruct, Bit-width=3-bit2025.08 | 38.71 | |
| VLMQBase Model=LLaVA-OneVision-7B, Bit-width=2-bit, Group Size=1282025.08 | 38.66 | |
| VLMQModel=Qwen2-VL-2B-Instruct, Bit-width=3-bit, Importance Factor Configuration=Default (ℓ1-norm)2025.08 | 37.94 | |
| GPTAQBase Model=Qwen2-VL-7B-Instruct, Bit-width=2-bit, Group Size=1282025.08 | 37.91 | |
| GPTQModel=Qwen2-VL-2B-Instruct, Bit-width=3-bit2025.08 | 33.25 | |
| AWQModel=Qwen2-VL-7B-Instruct, Bit-width=3-bit2025.08 | 30.2 | |
| MBQModel=Qwen2-VL-7B-Instruct, Bit-width=3-bit2025.08 | 27.23 | |
| VLMQModel=LLaVA-OneVision-0.5B, Bit-width=3-bit, Importance Factor Configuration=‡ (unspecified variant)2025.08 | 20.42 | |
| GPTQModel=LLaVA-OneVision-0.5B, Bit-width=3-bit2025.08 | 13.4 | |
| MBQBase Model=Qwen2-VL-7B-Instruct, Bit-width=2-bit, Group Size=1282025.08 | 12.46 | |
| AWQBase Model=Qwen2-VL-7B-Instruct, Bit-width=2-bit, Group Size=1282025.08 | 11.58 | |
| MBQModel=Qwen2-VL-2B-Instruct, Bit-width=3-bit2025.08 | 10.22 | |
| AWQModel=Qwen2-VL-2B-Instruct, Bit-width=3-bit2025.08 | 9.65 | |
| GPTAQModel=LLaVA-OneVision-0.5B, Bit-width=3-bit2025.08 | 9.2 |