Multiple Choice Question Answering on MC (test)
78MC AvgBaseline
Evaluation Results
| Method | Links | |
|---|---|---|
| BaselineModel=Kimi-K2-Instruct-W4A16, Compression=None2025.10 | 78 | |
| REAPModel=Kimi-K2-Instruct-W4A16, Compression=25%2025.10 | 77.3 | |
| BaselineModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=None2025.10 | 75 | |
| REAPModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=25%2025.10 | 74.8 | |
| Qwen3Sparsity=Full, Model=Qwen32026.03 | 73.7 | |
| FullModel=Qwen3, Ratio=0%, Zero-shot=true2026.03 | 73.7 | |
| EANModel=Qwen3, Ratio=25%, Zero-shot=true2026.03 | 73.6 | |
| SEERModel=Qwen3, Ratio=25%, Zero-shot=true2026.03 | 73.4 | |
| FrequencyModel=Qwen3, Ratio=25%, Zero-shot=true2026.03 | 73.2 | |
| REAPModel=Qwen3, Ratio=25%, Zero-shot=true2026.03 | 72.5 | |
| AIMERModel=Qwen3, Ratio=25%, Zero-shot=true2026.03 | 72.3 | |
| REAPSparsity=25%, Allocation Protocol=UNI2026.03 | 70.6 | |
| EANModel=Kimi-K2-Instruct-W4A16, Compression=25%2025.10 | 70.3 | |
| EANModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=25%2025.10 | 70.2 | |
| EANModel=Qwen3, Ratio=50%, Zero-shot=true2026.03 | 69.7 | |
| REAPModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=50%2025.10 | 69.2 | |
| RandomModel=Qwen3, Ratio=25%, Zero-shot=true2026.03 | 68.4 | |
| MagnitudeModel=Qwen3, Ratio=25%, Zero-shot=true2026.03 | 67 | |
| REAPSparsity=25%, Allocation Protocol=ESAP2026.03 | 66.5 | |
| REAPModel=Kimi-K2-Instruct-W4A16, Compression=50%2025.10 | 64.3 | |
| AIMERModel=Qwen3, Ratio=50%, Zero-shot=true2026.03 | 63.8 | |
| EANSparsity=25%, Allocation Protocol=ESAP2026.03 | 63.4 | |
| EANSparsity=25%, Allocation Protocol=UNI2026.03 | 62.8 | |
| REAPModel=Qwen3, Ratio=50%, Zero-shot=true2026.03 | 62.8 | |
| FrequencyModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=25%2025.10 | 60.6 | |
| FrequencyModel=Kimi-K2-Instruct-W4A16, Compression=25%2025.10 | 60.4 | |
| REAPSparsity=50%, Allocation Protocol=UNI2026.03 | 59.6 | |
| EANModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=50%2025.10 | 59.1 | |
| EANModel=Kimi-K2-Instruct-W4A16, Compression=50%2025.10 | 58.7 | |
| REAPSparsity=50%, Allocation Protocol=ESAP2026.03 | 58.5 | |
| SEERSparsity=25%, Allocation Protocol=ESAP2026.03 | 57.7 | |
| FreqSparsity=25%, Allocation Protocol=ESAP2026.03 | 57.2 | |
| SEERSparsity=25%, Allocation Protocol=UNI2026.03 | 56.1 | |
| FreqSparsity=25%, Allocation Protocol=UNI2026.03 | 55.8 | |
| EANSparsity=50%, Allocation Protocol=ESAP2026.03 | 51.8 | |
| EANSparsity=50%, Allocation Protocol=UNI2026.03 | 51.6 | |
| FreqSparsity=50%, Allocation Protocol=ESAP2026.03 | 51 | |
| FrequencyModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=50%2025.10 | 50.6 | |
| SEERSparsity=50%, Allocation Protocol=UNI2026.03 | 45.5 | |
| FreqSparsity=50%, Allocation Protocol=UNI2026.03 | 45 | |
| SEERSparsity=50%, Allocation Protocol=ESAP2026.03 | 44.6 | |
| FrequencyModel=Kimi-K2-Instruct-W4A16, Compression=50%2025.10 | 43.9 | |
| FrequencyModel=Qwen3, Ratio=50%, Zero-shot=true2026.03 | 43.7 | |
| SEERModel=Qwen3, Ratio=50%, Zero-shot=true2026.03 | 43.6 | |
| RandomModel=Qwen3, Ratio=50%, Zero-shot=true2026.03 | 40.9 | |
| MagnitudeModel=Qwen3, Ratio=50%, Zero-shot=true2026.03 | 37.2 |