Multimodal Evaluation on MME Chinese
60.37Overall ScoreFull Prec
Evaluation Results
| Method | Links | |
|---|---|---|
| Full PrecModel=Qwen2.5-VL-32B-Instruct, Bit-width=Full Precision2025.08 | 60.37 | |
| Full PrecModel=Qwen2-VL-7B-Instruct, Bit-width=Full Precision2025.08 | 56.09 | |
| Full PrecModel=LLaVA-OneVision-7B, Bit-width=Full Precision2025.08 | 53.93 | |
| Full PrecModel=Qwen2.5-VL-7B-Instruct, Bit-width=Full Precision2025.08 | 52.98 | |
| GPTQModel=Qwen2.5-VL-32B-Instruct, Bit-width=3-bit2025.08 | 47.76 | |
| VLMQModel=Qwen2-VL-7B-Instruct, Quantization Bit=INT42025.08 | 47.73 | |
| GPTAQModel=Qwen2-VL-7B-Instruct, Quantization Bit=INT42025.08 | 47.3 | |
| VLMQModel=Qwen2-VL-7B-Instruct, Bit-width=3-bit, Importance Factor Configuration=Default (ℓ1-norm)2025.08 | 47.25 | |
| VLMQModel=LLaVA-OneVision-7B, Bit-width=3-bit, Importance Factor Configuration=token-level ℓ2-norm2025.08 | 45.94 | |
| VLMQModel=LLaVA-OneVision-7B, Bit-width=3-bit, Importance Factor Configuration=Default (ℓ1-norm)2025.08 | 45.41 | |
| GPTQModel=Qwen2-VL-7B-Instruct, Quantization Bit=INT42025.08 | 44.03 | |
| GPTAQModel=Qwen2-VL-7B-Instruct, Bit-width=3-bit2025.08 | 43.96 | |
| Full PrecModel=Qwen2-VL-2B-Instruct, Bit-width=Full Precision2025.08 | 43.59 | |
| VLMQModel=Qwen2.5-VL-32B-Instruct, Bit-width=3-bit, Importance Factor Configuration=Default (ℓ1-norm)2025.08 | 43.45 | |
| GPTAQModel=LLaVA-OneVision-7B, Bit-width=3-bit2025.08 | 42.94 | |
| GPTAQModel=Qwen2.5-VL-32B-Instruct, Bit-width=3-bit2025.08 | 42.67 | |
| GPTQModel=LLaVA-OneVision-7B, Bit-width=3-bit2025.08 | 42.59 | |
| VLMQModel=Qwen2.5-VL-7B-Instruct, Bit-width=3-bit, Importance Factor Configuration=Default (ℓ1-norm)2025.08 | 37.89 | |
| GPTAQModel=Qwen2.5-VL-7B-Instruct, Bit-width=3-bit2025.08 | 36.49 | |
| VLMQBase Model=Qwen2-VL-7B-Instruct, Bit-width=2-bit, Group Size=128, Precursor Algorithm=GPTQ2025.08 | 35.59 | |
| GPTQBase Model=Qwen2-VL-7B-Instruct, Bit-width=2-bit, Group Size=1282025.08 | 34.11 | |
| GPTAQBase Model=Qwen2-VL-7B-Instruct, Bit-width=2-bit, Group Size=1282025.08 | 33.8 | |
| VLMQModel=Qwen2-VL-2B-Instruct, Bit-width=3-bit, Importance Factor Configuration=Default (ℓ1-norm)2025.08 | 32.58 | |
| GPTAQBase Model=LLaVA-OneVision-7B, Bit-width=2-bit, Group Size=1282025.08 | 32.45 | |
| Full PrecModel=LLaVA-OneVision-0.5B, Bit-width=Full Precision2025.08 | 32.13 | |
| VLMQBase Model=LLaVA-OneVision-7B, Bit-width=2-bit, Group Size=1282025.08 | 31.38 | |
| GPTQBase Model=LLaVA-OneVision-7B, Bit-width=2-bit, Group Size=1282025.08 | 31.35 | |
| GPTQModel=Qwen2-VL-2B-Instruct, Bit-width=3-bit2025.08 | 30.83 | |
| VLMQBase Model=Qwen2.5-VL-7B-Instruct, Bit-width=2-bit, Group Size=128, Precursor Algorithm=GPTQ2025.08 | 30.34 | |
| GPTAQModel=Qwen2-VL-2B-Instruct, Bit-width=3-bit2025.08 | 28.71 | |
| GPTQModel=Qwen2-VL-7B-Instruct, Bit-width=3-bit2025.08 | 27.9 | |
| GPTQModel=Qwen2.5-VL-7B-Instruct, Bit-width=3-bit2025.08 | 27.33 | |
| GPTQModel=LLaVA-OneVision-0.5B, Bit-width=3-bit2025.08 | 21.63 | |
| MBQModel=Qwen2-VL-7B-Instruct, Bit-width=3-bit2025.08 | 21.35 | |
| AWQModel=Qwen2-VL-7B-Instruct, Bit-width=3-bit2025.08 | 20.03 | |
| VLMQModel=LLaVA-OneVision-0.5B, Bit-width=3-bit, Importance Factor Configuration=‡ (unspecified variant)2025.08 | 15.08 | |
| GPTQBase Model=Qwen2.5-VL-7B-Instruct, Bit-width=2-bit, Group Size=1282025.08 | 13.89 | |
| GPTAQModel=LLaVA-OneVision-0.5B, Bit-width=3-bit2025.08 | 11.27 | |
| AWQModel=Qwen2-VL-2B-Instruct, Bit-width=3-bit2025.08 | 8.48 | |
| MBQBase Model=Qwen2-VL-7B-Instruct, Bit-width=2-bit, Group Size=1282025.08 | 7.94 | |
| MBQModel=Qwen2-VL-2B-Instruct, Bit-width=3-bit2025.08 | 7.79 | |
| AWQBase Model=Qwen2-VL-7B-Instruct, Bit-width=2-bit, Group Size=1282025.08 | 7.47 | |
| GPTAQBase Model=Qwen2.5-VL-7B-Instruct, Bit-width=2-bit, Group Size=1282025.08 | 3.6 |