Commonsense Reasoning on Winogrande (Accuracy and Standard Deviation)
74.9AccuracyFP
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| FPModel=LLADA, Precision=Full Precision2026.06 | 74.9 | — | — | |
| FPModel=LLADA-1.5, Precision=Full Precision2026.06 | 74.8 | — | — | |
| FPModel=DREAM, Precision=Full Precision2026.06 | 74.5 | — | — | |
| STaR-QuantModel=LLADA-1.5, Quantization=W8A82026.06 | 73.98 | — | — | |
| Llama-3.1-8BArchitecture=Transformer, Tokens (B)=15000, Evaluation Protocol=zero-shot2025.05 | 73.9 | — | — | |
| RecurrentGemma-9BArchitecture=Subquadratic (Pretrained), Tokens (B)=2000, Evaluation Protocol=zero-shot2025.05 | 73.7 | — | — | |
| LoLCATs-8BArchitecture=Subquadratic (Distilled), Tokens (B)=0.04, Evaluation Protocol=zero-shot2025.05 | 73.6 | — | — | |
| LoLA-8B (ours)Architecture=Subquadratic (Distilled), Tokens (B)=0.04, Evaluation Protocol=zero-shot, η=64, λ=642025.05 | 73.6 | — | — | |
| STaR-QuantModel=DREAM, Quantization=W8A82026.06 | 73.5 | — | — | |
| Llamba-8BArchitecture=Subquadratic (Distilled), Tokens (B)=12, Evaluation Protocol=zero-shot2025.05 | 73.3 | — | — | |
| Griffin 7BArchitecture=Subquadratic (Pretrained), Tokens (B)=300, Evaluation Protocol=zero-shot2025.05 | 72.6 | — | — | |
| Mamba2-8BArchitecture=Subquadratic (Pretrained), Tokens (B)=3500, Evaluation Protocol=zero-shot2025.05 | 71.6 | — | — | |
| RWKV-6 (W2.1) 7BArchitecture=Subquadratic (Pretrained), Tokens (B)=1420, Evaluation Protocol=zero-shot2025.05 | 70 | — | — | |
| Hawk 7BArchitecture=Subquadratic (Pretrained), Tokens (B)=300, Evaluation Protocol=zero-shot2025.05 | 69.9 | — | — | |
| SparseGPTBackbone=Llama-2-7B, Sparsity=50%, Evaluation Protocol=Zero-shot2025.10 | 69.77 | — | — | |
| L-ADMMBackbone=Llama-2-7B, Sparsity=50%, Evaluation Protocol=Zero-shot2025.10 | 69.3 | — | — | |
| SparseLLMBackbone=Llama-2-7B, Sparsity=50%, Evaluation Protocol=Zero-shot2025.10 | 69.3 | — | — | |
| Falcon3-Mamba-7BArchitecture=Subquadratic (Pretrained), Tokens (B)=7300, Evaluation Protocol=zero-shot2025.05 | 69.1 | — | — | |
| DenseBackbone=Llama-2-7B, Sparsity=0%, Evaluation Protocol=Zero-shot2025.10 | 69.06 | — | — | |
| ALPSBackbone=Llama-2-7B, Sparsity=50%, Evaluation Protocol=Zero-shot2025.10 | 68.98 | — | — | |
| STaR-QuantModel=LLADA, Quantization=W8A82026.06 | 68.72 | — | — | |
| Wanda + GRAILSparsity=20%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot, Calibration Dataset=C42026.02 | 68.43 | — | — | |
| Mamba-8BArchitecture=Subquadratic (Pretrained), Tokens (B)=1100, Evaluation Protocol=zero-shot2025.05 | 68.3 | — | — | |
| Wanda++Sparsity=20%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot, Calibration Dataset=C42026.02 | 68.27 | — | — | |
| SlimGPT + GRAILSparsity=20%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot, Calibration Dataset=C42026.02 | 68.19 | — | — | |
| Few-shotModel=Qwen3-8B, Training Size=1002026.02 | 67.84 | 0.13 | — | |
| Wanda++ + GRAILSparsity=20%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot, Calibration Dataset=C42026.02 | 67.72 | — | — | |
| WandaBackbone=Llama-2-7B, Sparsity=50%, Evaluation Protocol=Zero-shot2025.10 | 67.17 | — | — | |
| ZipLMSparsity=20%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot, Calibration Dataset=C42026.02 | 66.85 | — | — | |
| SAFEBackbone=Llama-2-7B, Sparsity=50%, Evaluation Protocol=Zero-shot2025.10 | 66.77 | — | — | |
| L-ADMMBackbone=Llama-2-7B, Sparsity=60%, Evaluation Protocol=Zero-shot2025.10 | 66.61 | — | — | |
| ELSABackbone=Llama-2-7B, Sparsity=50%, Evaluation Protocol=Zero-shot2025.10 | 66.54 | — | — | |
| FLAPSparsity=20%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot, Calibration Dataset=C42026.02 | 66.38 | — | — | |
| ALPSBackbone=Llama-2-7B, Sparsity=60%, Evaluation Protocol=Zero-shot2025.10 | 66.29 | — | — | |
| SparseLLMBackbone=Llama-2-7B, Sparsity=60%, Evaluation Protocol=Zero-shot2025.10 | 66.14 | — | — | |
| FLAP + GRAILSparsity=20%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot, Calibration Dataset=C42026.02 | 66.06 | — | — | |
| SparseGPTBackbone=Llama-2-7B, Sparsity=60%, Evaluation Protocol=Zero-shot2025.10 | 65.51 | — | — | |
| WandaSparsity=20%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot, Calibration Dataset=C42026.02 | 64.17 | — | — | |
| WandaBackbone=Llama-2-7B, Sparsity=60%, Evaluation Protocol=Zero-shot2025.10 | 64.01 | — | — | |
| CLIMBSize=950M2025.04 | 63.54 | — | — | |
| ELSABackbone=Llama-2-7B, Sparsity=60%, Evaluation Protocol=Zero-shot2025.10 | 63.38 | — | — | |
| MagnitudeBackbone=Llama-2-7B, Sparsity=50%, Evaluation Protocol=Zero-shot2025.10 | 63.22 | — | — | |
| SAFEBackbone=Llama-2-7B, Sparsity=60%, Evaluation Protocol=Zero-shot2025.10 | 62.98 | — | — | |
| ALPSBackbone=Llama-2-7B, Sparsity=70%, Evaluation Protocol=Zero-shot2025.10 | 61.72 | — | — | |
| AMD-OLMoSize=1.2B2025.04 | 61.64 | — | — | |
| L-ADMMBackbone=Llama-2-7B, Sparsity=70%, Evaluation Protocol=Zero-shot2025.10 | 60.77 | — | — | |
| Llama-3.2Size=1.2B2025.04 | 60.69 | — | — | |
| FLAPSparsity=50%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot, Calibration Dataset=C42026.02 | 59.83 | — | — | |
| FLAP + GRAILSparsity=50%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot, Calibration Dataset=C42026.02 | 59.83 | — | — | |
| CCGQAParameters=1B, Training Tokens=300B, Architecture=Dense, Evaluation Protocol=Zero-shot, KV-cache compression=1/42025.10 | 59.7 | — | — | |
| TinyLlamaSize=1.1B2025.04 | 59.12 | — | — | |
| ELSABackbone=Llama-2-7B, Sparsity=70%, Evaluation Protocol=Zero-shot2025.10 | 58.64 | — | — | |
| SlimGPT + GRAILSparsity=50%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot, Calibration Dataset=C42026.02 | 58.64 | — | — | |
| Mamba2-Llama3-8B (L3.1-Instr.)Architecture=Subquadratic (Distilled), Tokens (B)=20, Evaluation Protocol=zero-shot2025.05 | 58.6 | — | — | |
| CLIMBSize=350M2025.04 | 57.93 | — | — | |
| SparseGPTBackbone=Llama-2-7B, Sparsity=70%, Evaluation Protocol=Zero-shot2025.10 | 57.7 | — | — | |
| MLAParameters=1B, Training Tokens=300B, Architecture=Dense, Evaluation Protocol=Zero-shot, KV-cache compression=1/42025.10 | 57.6 | — | — | |
| SparseLLMBackbone=Llama-2-7B, Sparsity=70%, Evaluation Protocol=Zero-shot2025.10 | 57.46 | — | — | |
| GQAParameters=1B, Training Tokens=300B, Architecture=Dense, Evaluation Protocol=Zero-shot, KV-cache compression=1/42025.10 | 56.9 | — | — | |
| MHAParameters=1B, Training Tokens=300B, Architecture=Dense, Evaluation Protocol=Zero-shot, KV-cache compression=1/12025.10 | 56.8 | — | — | |
| SmolLMSize=360M2025.04 | 56.75 | — | — | |
| CCAParameters=1B, Training Tokens=300B, Architecture=Dense, Evaluation Protocol=Zero-shot, KV-cache compression=1/42025.10 | 56.7 | — | — | |
| Qwen2.5Size=490M2025.04 | 56.59 | — | — | |
| ZipLMSparsity=50%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot, Calibration Dataset=C42026.02 | 55.64 | — | — | |
| Wanda + GRAILSparsity=50%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot, Calibration Dataset=C42026.02 | 54.46 | — | — | |
| Hedgehog-8B (Llama-3)Architecture=Subquadratic (Distilled), Tokens (B)=0.04, Evaluation Protocol=zero-shot2025.05 | 54.3 | — | — | |
| SlimGPTSparsity=20%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot, Calibration Dataset=C42026.02 | 53.59 | — | — | |
| ELSABackbone=Llama-2-7B, Sparsity=80%, Evaluation Protocol=Zero-shot2025.10 | 53.2 | — | — | |
| MagnitudeBackbone=Llama-2-7B, Sparsity=60%, Evaluation Protocol=Zero-shot2025.10 | 53.12 | — | — | |
| SAFEBackbone=Llama-2-7B, Sparsity=70%, Evaluation Protocol=Zero-shot2025.10 | 53.12 | — | — | |
| SlimGPTSparsity=50%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot, Calibration Dataset=C42026.02 | 52.96 | — | — | |
| Wanda++ + GRAILSparsity=50%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot, Calibration Dataset=C42026.02 | 52.8 | — | — | |
| ROASTModel=Gemma3-1B, Training Size=1000, Aggregation=no-group2026.02 | 52.56 | 0.01 | — | |
| ROASTModel=Qwen3-0.6B, Training Size=1000, Aggregation=group2026.02 | 52.36 | 0 | — | |
| ELSABackbone=Llama-2-7B, Sparsity=90%, Evaluation Protocol=Zero-shot2025.10 | 51.7 | — | — | |
| MagnitudeBackbone=Llama-2-7B, Sparsity=80%, Evaluation Protocol=Zero-shot2025.10 | 51.62 | — | — | |
| L-ADMMBackbone=Llama-2-7B, Sparsity=80%, Evaluation Protocol=Zero-shot2025.10 | 51.22 | — | — | |
| ALPSBackbone=Llama-2-7B, Sparsity=80%, Evaluation Protocol=Zero-shot2025.10 | 50.91 | — | — | |
| SparseGPTBackbone=Llama-2-7B, Sparsity=90%, Evaluation Protocol=Zero-shot2025.10 | 50.59 | — | — | |
| MagnitudeBackbone=Llama-2-7B, Sparsity=90%, Evaluation Protocol=Zero-shot2025.10 | 50.12 | — | — | |
| Wanda++Sparsity=50%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot, Calibration Dataset=C42026.02 | 50.12 | — | — | |
| BaselineModel=Qwen3-0.6B2026.02 | 49.8 | 0.59 | — | |
| SAFEBackbone=Llama-2-7B, Sparsity=80%, Evaluation Protocol=Zero-shot2025.10 | 49.8 | — | — | |
| WandaBackbone=Llama-2-7B, Sparsity=90%, Evaluation Protocol=Zero-shot2025.10 | 49.33 | — | — | |
| MagnitudeBackbone=Llama-2-7B, Sparsity=70%, Evaluation Protocol=Zero-shot2025.10 | 49.25 | — | — | |
| WandaBackbone=Llama-2-7B, Sparsity=80%, Evaluation Protocol=Zero-shot2025.10 | 49.25 | — | — | |
| SparseGPTBackbone=Llama-2-7B, Sparsity=80%, Evaluation Protocol=Zero-shot2025.10 | 49.01 | — | — | |
| WandaBackbone=Llama-2-7B, Sparsity=70%, Evaluation Protocol=Zero-shot2025.10 | 48.86 | — | — | |
| SparseLLMBackbone=Llama-2-7B, Sparsity=80%, Evaluation Protocol=Zero-shot2025.10 | 48.86 | — | — | |
| WandaSparsity=50%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot, Calibration Dataset=C42026.02 | 48.38 | — | — | |
| SAFEBackbone=Llama-2-7B, Sparsity=90%, Evaluation Protocol=Zero-shot2025.10 | 47.83 | — | — | |
| L-ADMMBackbone=Llama-2-7B, Sparsity=90%, Evaluation Protocol=Zero-shot2025.10 | 47.51 | — | — | |
| SparseLLMBackbone=Llama-2-7B, Sparsity=90%, Evaluation Protocol=Zero-shot2025.10 | 46.96 | — | — | |
| ALPSBackbone=Llama-2-7B, Sparsity=90%, Evaluation Protocol=Zero-shot2025.10 | 46.65 | — | — | |
| Acc (Ours)Pruning Method=Accuracy-based relevance score, Base Model=LLaMA3-8B, Calibration Dataset Size=1,500-instance2026.05 | — | — | 60.14 | |
| BaseModel=LLaMA-3.1-8B, Compression=0%, Mode=Zero-shot2025.10 | — | — | 73.56 | |
| BaselineModel=Qwen2.52026.04 | — | — | 72.3 | |
| BaselineModel=LLaMA-3.12026.04 | — | — | 73.8 | |
| BaselineModel=Qwen32026.04 | — | — | 68.7 | |
| BaselineBackbone=Qwen1.5-MoE-A2.7B, Type=–, Storage=28.67GB, Evaluation Protocol=zero-shot2026.06 | — | — | 69.77 |