Zero-shot Reasoning on WinoGrande
80.51AccuracyFP16
Evaluation results
| Method | Links | |
|---|---|---|
| FP16Backbone=Llama-3-70B, Bits=FP162026.06 | 80.51 | |
| LiftQuantBackbone=Llama-3-70B, Bits=2.012026.06 | 78.53 | |
| FP16Backbone=Llama-2-70B, Bits=FP162026.06 | 77.98 | |
| VPTQBackbone=Llama-3-70B, Bits=2.022026.06 | 77.9 | |
| LiftQuantBackbone=Llama-2-70B, Bits=2.412026.06 | 77.19 | |
| QTIPBackbone=Llama-2-70B, Bits=2.002026.06 | 76.19 | |
| QuIP#Backbone=Llama-2-70B, Bits=2.002026.06 | 75.77 | |
| LiftQuantBackbone=Llama-2-70B, Bits=2.012026.06 | 75.69 | |
| FP16Model=Mistral, Precision=FP16, Sequence Length=20482025.07 | 75.22 | |
| EQAT-g64Backbone=Llama-2-70B, Bits=2.262026.06 | 74.59 | |
| EQAT-g64Backbone=Llama-3-70B, Bits=2.252026.06 | 74.03 | |
| FP16Model=LLaMA-3-8B, Precision=FP16, Sequence Length=20482025.07 | 73.7 | |
| OASISModel=Mistral, Precision=W4A4, Sequence Length=20482025.07 | 73.4 | |
| OASIS-SModel=Mistral, Precision=W4A4, Sequence Length=20482025.07 | 72.97 | |
| FP16Backbone=Llama-3-8B, Bits=FP162026.06 | 72.61 | |
| QuaRotModel=Mistral, Precision=W4A4, Sequence Length=20482025.07 | 72.34 | |
| FP16Backbone=Llama-2-13B, Bits=FP162026.06 | 72.22 | |
| AtomModel=Mistral, Precision=W4A4, Sequence Length=2048, Group Size=1282025.07 | 72.03 | |
| OASISModel=LLaMA-3-8B, Precision=W4A4, Sequence Length=20482025.07 | 71.27 | |
| LiftQuantBackbone=Llama-2-13B, Bits=2.422026.06 | 71.27 | |
| OASIS-SModel=LLaMA-3-8B, Precision=W4A4, Sequence Length=20482025.07 | 70.56 | |
| OASISModel=Mistral, Precision=W4A3, Sequence Length=20482025.07 | 70.56 | |
| QTIPBackbone=Llama-2-13B, Bits=2.002026.06 | 70.4 | |
| LiftQuantBackbone=Llama-2-13B, Bits=2.022026.06 | 70.01 | |
| BaselineRatio=1.0, Memory Footprint (GB)=12.6GB, Zero-shot=true2026.04 | 70 | |
| FP16Backbone=Llama-2-7B, Bits=FP162026.06 | 69.22 | |
| QuIP#Backbone=Llama-2-13B, Bits=2.002026.06 | 69.06 | |
| Llama2-7BBackbone=Llama2-7B2026.02 | 69 | |
| FP16Model=LLaMA-2-7B, Precision=FP16, Sequence Length=20482025.07 | 68.98 | |
| DensePrune Rate=0%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 68.43 | |
| EQAT-g64Backbone=Llama-2-13B, Bits=2.252026.06 | 68.36 | |
| LiftQuantBackbone=Llama-2-7B, Bits=2.022026.06 | 68.27 | |
| LiftQuantBackbone=Llama-3-8B, Bits=2.022026.06 | 68.03 | |
| Swift-SVDRatio=0.8, Memory Footprint (GB)=10.1 GB, Zero-shot=true2026.04 | 68 | |
| Swift-SVD*Ratio=0.8, Memory Footprint (GB)=10.1 GB, Zero-shot=true2026.04 | 68 | |
| LiftQuantBackbone=Llama-2-7B, Bits=2.422026.06 | 67.72 | |
| EQAT-g64Backbone=Llama-3-8B, Bits=2.252026.06 | 67.72 | |
| COMPOTBackbone=Llama2-7B, Training-free=true, CR=0.22026.02 | 67.6 | |
| QTIPBackbone=Llama-2-7B, Bits=2.002026.06 | 67.25 | |
| OASIS-SModel=Mistral, Precision=W4A3, Sequence Length=20482025.07 | 66.93 | |
| OASISModel=LLaMA-3-8B, Precision=W4A3, Sequence Length=20482025.07 | 66.22 | |
| EQAT-g64Backbone=Llama-2-7B, Bits=2.252026.06 | 65.98 | |
| VPTQBackbone=Llama-3-8B, Bits=2.072026.06 | 65.92 | |
| QuIP#Backbone=Llama-2-7B, Bits=2.002026.06 | 65.67 | |
| OASISModel=LLaMA-2-7B, Precision=W4A4, Sequence Length=20482025.07 | 65.51 | |
| OASIS-SModel=LLaMA-3-8B, Precision=W4A3, Sequence Length=20482025.07 | 65.03 | |
| OASIS-SModel=LLaMA-2-7B, Precision=W4A4, Sequence Length=20482025.07 | 64.8 | |
| OASISModel=LLaMA-2-7B, Precision=W4A3, Sequence Length=20482025.07 | 64.17 | |
| ASVDRatio=0.8, Memory Footprint (GB)=10.1 GB, Zero-shot=true2026.04 | 64 | |
| OASIS-SModel=LLaMA-2-7B, Precision=W4A3, Sequence Length=20482025.07 | 63.93 | |
| AtomModel=Mistral, Precision=W4A3, Sequence Length=2048, Group Size=1282025.07 | 63.73 | |
| COMPOTBackbone=Llama2-7B, Training-free=true, CR=0.42026.02 | 63.5 | |
| SliceGPTPrune Rate=30%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 63.22 | |
| QuaRotModel=LLaMA-2-7B, Precision=W4A4, Sequence Length=20482025.07 | 63.06 | |
| QuaRotModel=LLaMA-3-8B, Precision=W4A4, Sequence Length=20482025.07 | 63.06 | |
| AtomModel=LLaMA-2-7B, Precision=W4A4, Sequence Length=2048, Group Size=1282025.07 | 62.75 | |
| Dobi-SVD(w)Ratio=0.8, Memory Footprint (GB)=10.1 GB, Zero-shot=true2026.04 | 62 | |
| Swift-SVDRatio=0.6, Memory Footprint (GB)=7.7 GB, Zero-shot=true2026.04 | 62 | |
| Swift-SVD*Ratio=0.6, Memory Footprint (GB)=7.7 GB, Zero-shot=true2026.04 | 62 | |
| PG pruningPrune Rate=30%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 61.6 | |
| AtomModel=LLaMA-3-8B, Precision=W4A4, Sequence Length=2048, Group Size=1282025.07 | 61.13 | |
| SVD-LLM(W)Ratio=0.8, Memory Footprint (GB)=10.1 GB, Zero-shot=true2026.04 | 61 | |
| BonsaiPrune Rate=30%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 60.93 | |
| AtomModel=LLaMA-3-8B, Precision=W4A3, Sequence Length=2048, Group Size=1282025.07 | 60.87 | |
| SliceGPTPrune Rate=40%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 60.38 | |
| Wanda-spPrune Rate=30%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 60.3 | |
| AtomModel=LLaMA-2-7B, Precision=W4A3, Sequence Length=2048, Group Size=1282025.07 | 59.5 | |
| Nirvana (Ours)Setting=Zero-shot, Model Size=1.3B parameters2025.10 | 59.48 | |
| Nirvana-noTriggerSetting=Zero-shot, Model Size=1.3B parameters2025.10 | 59.27 | |
| Dobi-SVDBackbone=Llama2-7B, Training-free=false, CR=0.22026.02 | 58.5 | |
| Gated DeltaNet-H1Setting=Zero-shot, Model Size=1.3B parameters2025.10 | 58.4 | |
| Dobi-SVD(w)Ratio=0.6, Memory Footprint (GB)=7.7 GB, Zero-shot=true2026.04 | 58 | |
| Gated DeltaNet-H2Setting=Zero-shot, Model Size=1.3B parameters2025.10 | 57.77 | |
| Gated DeltaNetSetting=Zero-shot, Model Size=1.3B parameters2025.10 | 57.45 | |
| SliceGPTPrune Rate=50%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 56.27 | |
| PG pruningPrune Rate=40%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 55.72 | |
| SambaSetting=Zero-shot, Model Size=1.3B parameters2025.10 | 55.56 | |
| BonsaiPrune Rate=40%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 55.41 | |
| Mamba2Setting=Zero-shot, Model Size=1.3B parameters2025.10 | 55.24 | |
| SVD-LLM(W)Ratio=0.6, Memory Footprint (GB)=7.7 GB, Zero-shot=true2026.04 | 55 | |
| Dobi-SVDBackbone=Llama2-7B, Training-free=false, CR=0.42026.02 | 54.4 | |
| RetNetSetting=Zero-shot, Model Size=1.3B parameters2025.10 | 54.14 | |
| LLM-PrunerPrune Rate=30%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 54.06 | |
| Transformer++Setting=Zero-shot, Model Size=1.3B parameters2025.10 | 53.51 | |
| COMPOTBackbone=Llama2-7B, Training-free=true, CR=0.62026.02 | 53.4 | |
| DeltaNetSetting=Zero-shot, Model Size=1.3B parameters2025.10 | 53.35 | |
| BonsaiPrune Rate=50%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 53.12 | |
| Swift-SVD*Ratio=0.4, Memory Footprint (GB)=5.3 GB, Zero-shot=true2026.04 | 53 | |
| MambaSetting=Zero-shot, Model Size=1.3B parameters2025.10 | 52.95 | |
| Wanda-spPrune Rate=50%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 52.8 | |
| HGRN2Setting=Zero-shot, Model Size=1.3B parameters2025.10 | 52.8 | |
| PG pruningPrune Rate=50%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 52.64 | |
| Wanda-spPrune Rate=40%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 52.17 | |
| Swift-SVDRatio=0.4, Memory Footprint (GB)=5.3 GB, Zero-shot=true2026.04 | 52 | |
| LLM-PrunerPrune Rate=40%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 51.7 | |
| Dobi-SVDBackbone=Llama2-7B, Training-free=false, CR=0.62026.02 | 50.5 | |
| QuaRotModel=LLaMA-3-8B, Precision=W4A3, Sequence Length=20482025.07 | 50.2 | |
| LLM-PrunerPrune Rate=50%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 50.11 | |
| QuaRotModel=LLaMA-2-7B, Precision=W4A3, Sequence Length=20482025.07 | 49.49 | |
| QuaRotModel=Mistral, Precision=W4A3, Sequence Length=20482025.07 | 48.78 |