Commonsense Reasoning on Winogrande (ACC, TR)
76.09AccuracyFLAP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FLAPPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 76.09 | — | |
| RFID-MoEBackbone=Qwen2-57B-A14B, Ratio=40%, Evaluation Protocol=Zero-shot2026.02 | 76 | — | |
| MC-SMoEPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 75.14 | — | |
| SSQR-1%Avg. Bitwidth=4.445, Zero-shot=true2025.07 | 74.43 | — | |
| Mistral-7BBase Model=Mistral-7B2026.04 | 74.3 | — | |
| Qwen2-57B-A14BPruning ratio=0%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 74.27 | — | |
| RSPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 74.19 | — | |
| SSQR-5%Avg. Bitwidth=5.725, Zero-shot=true2025.07 | 74.03 | — | |
| RFID-MoEBackbone=Qwen3-30B-A3B-2507, Ratio=20%, Evaluation Protocol=Zero-shot2026.02 | 74 | — | |
| GPTQAvg. Bitwidth=4.125, Zero-shot=true2025.07 | 73.8 | — | |
| SSQR-2%Avg. Bitwidth=4.765, Zero-shot=true2025.07 | 73.8 | — | |
| BF16 BaselineAvg. Bitwidth=16.000, Zero-shot=true2025.07 | 73.72 | — | |
| HPTQAvg. Bitwidth=4.125, Zero-shot=true2025.07 | 73.56 | — | |
| SSQR-5%Avg. Bitwidth=4.725, Zero-shot=true2025.07 | 73.4 | — | |
| SSQR-2%Avg. Bitwidth=3.765, Zero-shot=true2025.07 | 73.24 | — | |
| SSQR-4%Avg. Bitwidth=5.405, Zero-shot=true2025.07 | 73.24 | — | |
| SSQR-4%Avg. Bitwidth=4.405, Zero-shot=true2025.07 | 73.24 | — | |
| MoNEPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 73.09 | — | |
| SSQR-3%Avg. Bitwidth=4.085, Zero-shot=true2025.07 | 73.09 | — | |
| SSQR-3%Avg. Bitwidth=5.085, Zero-shot=true2025.07 | 72.93 | — | |
| Llama-3-8BBase Model=Llama-3-8B2026.04 | 72.9 | — | |
| LoLCATsBase Model=Llama-3-8B, Healing Tokens (B)=0.042026.04 | 72.9 | — | |
| HPTQAvg. Bitwidth=3.125, Zero-shot=true2025.07 | 72.77 | — | |
| GPTQAvg. Bitwidth=3.125, Zero-shot=true2025.07 | 72.45 | — | |
| SSQR-1%Avg. Bitwidth=3.445, Zero-shot=true2025.07 | 72.45 | — | |
| RSPruning ratio=25%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 72.3 | — | |
| Liger-GLABase Model=Llama-3-8B, Healing Tokens (B)=0.022026.04 | 72 | — | |
| MoNEPruning ratio=25%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 71.98 | — | |
| Deepseek-V2-LitePruning ratio=0%, Backbone=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 71.51 | — | |
| MC-SMoEPruning ratio=25%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 71.43 | — | |
| MoNEPruning ratio=25%, Backbone=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 71.43 | — | |
| MoonlightPruning ratio=0%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 71.11 | — | |
| LoLCATsBase Model=Mistral-7B, Healing Tokens (B)=0.042026.04 | 71 | — | |
| Qwen3-30B-A3BPruning ratio=0%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 70.64 | — | |
| MoNEPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 70.48 | — | |
| MC-SMoEPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 70.4 | — | |
| SUPRABase Model=Mistral-7B, Healing Tokens (B)=1002026.04 | 70.3 | — | |
| RSPruning ratio=25%, Backbone=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 70.24 | — | |
| FLAPPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 70.17 | — | |
| RIDERSModel=GPT-3.52024.02 | 70.1 | 41.7 | |
| Liger-GLABase Model=Mistral-7B, Healing Tokens (B)=0.022026.04 | 70.1 | — | |
| RSPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 69.61 | — | |
| RFID-MoEBackbone=DeepSeekMoE-16B-Base, Ratio=40%, Evaluation Protocol=Zero-shot2026.02 | 69 | — | |
| OLMoEPruning ratio=0%, Backbone=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 68.9 | — | |
| MC-SMoEPruning ratio=25%, Backbone=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 68.67 | — | |
| FLAPPruning ratio=25%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 68.19 | — | |
| FLAPPruning ratio=25%, Backbone=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 67.72 | — | |
| RFID-MoEBackbone=Qwen1.5-MoE-A2.7B, Ratio=40%, Evaluation Protocol=Zero-shot2026.02 | 67 | — | |
| LayerBoostBase Model=Qwen3-4B, Healing Tokens (B)=0.042026.04 | 67 | — | |
| MoNEPruning ratio=25%, Backbone=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 66.46 | — | |
| SUPRABase Model=Llama-3-8B, Healing Tokens (B)=202026.04 | 65.8 | — | |
| MC-SMoEPruning ratio=25%, Backbone=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 65.75 | — | |
| Qwen3-4BBase Model=Qwen3-4B2026.04 | 65.7 | — | |
| CoTModel=GPT-3.52024.02 | 64.6 | 48.8 | |
| SSQR-5%Avg. Bitwidth=3.725, Zero-shot=true2025.07 | 64.25 | — | |
| HPTQAvg. Bitwidth=2.125, Zero-shot=true2025.07 | 63.69 | — | |
| Ettin-Dec-1BModel Size Category=XL, Parameters=908M-1.2B, Zero-shot=true, Closed-book=true2025.07 | 62.7 | — | |
| AngularPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 62.51 | — | |
| AngularPruning ratio=25%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 62.04 | — | |
| RIDERSModel=Mistral-7B2024.02 | 62 | 15.3 | |
| OLMo-1B-0724Model Size Category=XL, Parameters=908M-1.2B, Zero-shot=true, Closed-book=true2025.07 | 61.6 | — | |
| AngularPruning ratio=25%, Backbone=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 61.01 | — | |
| FLAPPruning ratio=25%, Backbone=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 60.93 | — | |
| Llama-3.2-1BModel Size Category=XL, Parameters=908M-1.2B, Zero-shot=true, Closed-book=true2025.07 | 60.6 | — | |
| RSPruning ratio=25%, Backbone=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 59.98 | — | |
| CoTModel=Mistral-7B2024.02 | 59.2 | 27.9 | |
| SmolLM2-360mModel Size Category=Large, Parameters=360-410M, Zero-shot=true, Closed-book=true2025.07 | 58.6 | — | |
| Mamba2-LlamaBase Model=Llama-3-8B, Healing Tokens (B)=202026.04 | 58.6 | — | |
| Ettin-Dec-400mModel Size Category=Large, Parameters=360-410M, Zero-shot=true, Closed-book=true2025.07 | 57.6 | — | |
| AngularPruning ratio=25%, Backbone=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 55.09 | — | |
| GPTQAvg. Bitwidth=2.125, Zero-shot=true2025.07 | 54.93 | — | |
| SSQR-3%Avg. Bitwidth=3.085, Zero-shot=true2025.07 | 54.54 | — | |
| SSQR-4%Avg. Bitwidth=3.405, Zero-shot=true2025.07 | 54.54 | — | |
| SSQR-2%Avg. Bitwidth=2.765, Zero-shot=true2025.07 | 54.3 | — | |
| SmolLM2-135mModel Size Category=Base, Parameters=135-160M, Zero-shot=true, Closed-book=true2025.07 | 53.7 | — | |
| Ettin-Dec-150mModel Size Category=Base, Parameters=135-160M, Zero-shot=true, Closed-book=true2025.07 | 53.7 | — | |
| Pythia-410mModel Size Category=Large, Parameters=360-410M, Zero-shot=true, Closed-book=true2025.07 | 53.6 | — | |
| Ettin-Dec-32mModel Size Category=XS, Parameters=32M, Zero-shot=true, Closed-book=true2025.07 | 53.1 | — | |
| Glauber-MN=32026.05 | 52.9 | — | |
| SSQR-1%Avg. Bitwidth=2.445, Zero-shot=true2025.07 | 52.8 | — | |
| (Gong et al., 2025)-M2026.05 | 52.6 | — | |
| Pythia-160mModel Size Category=Base, Parameters=135-160M, Zero-shot=true, Closed-book=true2025.07 | 52.4 | — | |
| GPPoM2-S Hyb.Architecture=Hybrid2026.04 | 51.9 | — | |
| B158Size=1.7B2026.05 | 51.22 | — | |
| Ettin-Dec-17mModel Size Category=XXS, Parameters=14-17M, Zero-shot=true, Closed-book=true2025.07 | 50.9 | — | |
| BF16Size=0.4B2026.05 | 50.75 | — | |
| GPT-2-M2026.05 | 50.7 | — | |
| +LGRSize=1.7B2026.05 | 50.51 | — | |
| DistilGPTModel Size Category=Small, Parameters=68-82M, Zero-shot=true, Closed-book=true2025.07 | 50.4 | — | |
| Pythia-14mModel Size Category=XXS, Parameters=14-17M, Zero-shot=true, Closed-book=true2025.07 | 50.3 | — | |
| +LGRSize=1.0B2026.05 | 50.28 | — | |
| Ettin-Dec-68mModel Size Category=Small, Parameters=68-82M, Zero-shot=true, Closed-book=true2025.07 | 50.1 | — | |
| +LGRSize=0.4B2026.05 | 50.04 | — | |
| GPT2-SSource=OpenAI2026.04 | 50 | — | |
| Glauber-MN=12026.05 | 49.9 | — | |
| B158Size=0.4B2026.05 | 49.8 | — | |
| AngularPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 49.57 | — | |
| BF16Size=1.7B2026.05 | 49.41 | — | |
| GPT2-SArchitecture=12MHA2026.04 | 49.4 | — | |
| SEDD-M2026.05 | 49 | — |