List your company
ResearchFormerly WizWand

Multiple Choice Question Answering on MC (test)

78MC AvgBaseline
35.56846.58457.668.616Oct 15, 2025Nov 9, 2025Dec 5, 2025Dec 31, 2025Jan 26, 2026Feb 21, 2026Mar 19, 2026
Updated 7mo ago

Evaluation results

MethodLinks
BaselineModel=Kimi-K2-Instruct-W4A16, Compression=None2025.1078
REAPModel=Kimi-K2-Instruct-W4A16, Compression=25%2025.1077.3
BaselineModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=None2025.1075
REAPModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=25%2025.1074.8
Qwen3Sparsity=Full, Model=Qwen32026.0373.7
FullModel=Qwen3, Ratio=0%, Zero-shot=true2026.0373.7
EANModel=Qwen3, Ratio=25%, Zero-shot=true2026.0373.6
SEERModel=Qwen3, Ratio=25%, Zero-shot=true2026.0373.4
FrequencyModel=Qwen3, Ratio=25%, Zero-shot=true2026.0373.2
REAPModel=Qwen3, Ratio=25%, Zero-shot=true2026.0372.5
AIMERModel=Qwen3, Ratio=25%, Zero-shot=true2026.0372.3
REAPSparsity=25%, Allocation Protocol=UNI2026.0370.6
EANModel=Kimi-K2-Instruct-W4A16, Compression=25%2025.1070.3
EANModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=25%2025.1070.2
EANModel=Qwen3, Ratio=50%, Zero-shot=true2026.0369.7
REAPModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=50%2025.1069.2
RandomModel=Qwen3, Ratio=25%, Zero-shot=true2026.0368.4
MagnitudeModel=Qwen3, Ratio=25%, Zero-shot=true2026.0367
REAPSparsity=25%, Allocation Protocol=ESAP2026.0366.5
REAPModel=Kimi-K2-Instruct-W4A16, Compression=50%2025.1064.3
AIMERModel=Qwen3, Ratio=50%, Zero-shot=true2026.0363.8
EANSparsity=25%, Allocation Protocol=ESAP2026.0363.4
EANSparsity=25%, Allocation Protocol=UNI2026.0362.8
REAPModel=Qwen3, Ratio=50%, Zero-shot=true2026.0362.8
FrequencyModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=25%2025.1060.6
FrequencyModel=Kimi-K2-Instruct-W4A16, Compression=25%2025.1060.4
REAPSparsity=50%, Allocation Protocol=UNI2026.0359.6
EANModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=50%2025.1059.1
EANModel=Kimi-K2-Instruct-W4A16, Compression=50%2025.1058.7
REAPSparsity=50%, Allocation Protocol=ESAP2026.0358.5
SEERSparsity=25%, Allocation Protocol=ESAP2026.0357.7
FreqSparsity=25%, Allocation Protocol=ESAP2026.0357.2
SEERSparsity=25%, Allocation Protocol=UNI2026.0356.1
FreqSparsity=25%, Allocation Protocol=UNI2026.0355.8
EANSparsity=50%, Allocation Protocol=ESAP2026.0351.8
EANSparsity=50%, Allocation Protocol=UNI2026.0351.6
FreqSparsity=50%, Allocation Protocol=ESAP2026.0351
FrequencyModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=50%2025.1050.6
SEERSparsity=50%, Allocation Protocol=UNI2026.0345.5
FreqSparsity=50%, Allocation Protocol=UNI2026.0345
SEERSparsity=50%, Allocation Protocol=ESAP2026.0344.6
FrequencyModel=Kimi-K2-Instruct-W4A16, Compression=50%2025.1043.9
FrequencyModel=Qwen3, Ratio=50%, Zero-shot=true2026.0343.7
SEERModel=Qwen3, Ratio=50%, Zero-shot=true2026.0343.6
RandomModel=Qwen3, Ratio=50%, Zero-shot=true2026.0340.9
MagnitudeModel=Qwen3, Ratio=50%, Zero-shot=true2026.0337.2