List your company
ResearchFormerly WizWand

Multimodal Understanding on MMMU zero-shot

73.67Zero-shot AccuracyBF16
35.533245.434155.33565.2359Feb 1, 2026
Updated 8mo ago

Evaluation results

MethodLinks
BF16Model=Qwen3-VL-30B-A3B-Instruct, Bit-width=BF16, Evaluation Protocol=Zero-shot2026.0273.67
GPTQModel=Qwen3-VL-30B-A3B-Instruct, Bit-width=W4, Evaluation Protocol=Zero-shot2026.0272.67
VEQ-MAModel=Qwen3-VL-30B-A3B-Instruct, Bit-width=W4, Evaluation Protocol=Zero-shot2026.0271.56
MBQModel=Qwen3-VL-30B-A3B-Instruct, Bit-width=W4, Evaluation Protocol=Zero-shot2026.0269.67
VEQ-MEModel=Qwen3-VL-30B-A3B-Instruct, Bit-width=W4, Evaluation Protocol=Zero-shot2026.0268.78
VEQ-MAModel=Qwen3-VL-30B-A3B-Instruct, Bit-width=W3, Evaluation Protocol=Zero-shot2026.0265.89
AWQModel=Qwen3-VL-30B-A3B-Instruct, Bit-width=W4, Evaluation Protocol=Zero-shot2026.0265.2
GPTQModel=Qwen3-VL-30B-A3B-Instruct, Bit-width=W3, Evaluation Protocol=Zero-shot2026.0264.89
RTNModel=Qwen3-VL-30B-A3B-Instruct, Bit-width=W4, Evaluation Protocol=Zero-shot2026.0263.11
VEQ-MEModel=Qwen3-VL-30B-A3B-Instruct, Bit-width=W3, Evaluation Protocol=Zero-shot2026.0260.56
AWQModel=Qwen3-VL-30B-A3B-Instruct, Bit-width=W3, Evaluation Protocol=Zero-shot2026.0258.89
RTNModel=Qwen3-VL-30B-A3B-Instruct, Bit-width=W3, Evaluation Protocol=Zero-shot2026.0257.33
BF16Model=Kimi-VL-Instruct, Bit-width=BF16, Evaluation Protocol=Zero-shot2026.0251.11
MBQModel=Qwen3-VL-30B-A3B-Instruct, Bit-width=W3, Evaluation Protocol=Zero-shot2026.0250.56
VEQ-MAModel=Kimi-VL-Instruct, Bit-width=W4, Evaluation Protocol=Zero-shot2026.0250.3
GPTQModel=Kimi-VL-Instruct, Bit-width=W4, Evaluation Protocol=Zero-shot2026.0250
VEQ-MEModel=Kimi-VL-Instruct, Bit-width=W4, Evaluation Protocol=Zero-shot2026.0249.22
AWQModel=Kimi-VL-Instruct, Bit-width=W4, Evaluation Protocol=Zero-shot2026.0249
MBQModel=Kimi-VL-Instruct, Bit-width=W4, Evaluation Protocol=Zero-shot2026.0248.89
RTNModel=Kimi-VL-Instruct, Bit-width=W4, Evaluation Protocol=Zero-shot2026.0248.44
VEQ-MEModel=Kimi-VL-Instruct, Bit-width=W3, Evaluation Protocol=Zero-shot2026.0244.56
VEQ-MAModel=Kimi-VL-Instruct, Bit-width=W3, Evaluation Protocol=Zero-shot2026.0242.56
AWQModel=Kimi-VL-Instruct, Bit-width=W3, Evaluation Protocol=Zero-shot2026.0241.78
GPTQModel=Kimi-VL-Instruct, Bit-width=W3, Evaluation Protocol=Zero-shot2026.0240.33
MBQModel=Kimi-VL-Instruct, Bit-width=W3, Evaluation Protocol=Zero-shot2026.0239.76
RTNModel=Kimi-VL-Instruct, Bit-width=W3, Evaluation Protocol=Zero-shot2026.0237