Zero-shot Reasoning on HellaSwag
76.3AccuracyLlama2-7B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Llama2-7BBackbone=Llama2-7B2026.02 | 76.3 | — | |
| SliceGPTPrune Rate=30%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 60.11 | — | |
| DensePrune Rate=0%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 57.17 | — | |
| LLaMA-2 7B (Dense)Backbone=LLaMA-2 7B, Pruning Method=Dense, Sparsity Pattern=None2024.09 | 57.03 | — | |
| BaselineRatio=1.0, Memory Footprint (GB)=12.6GB, Zero-shot=true2026.04 | 57 | — | |
| Nemotron-4 15B + MaskLLMBackbone=Nemotron-4 15B, Pruning Method=MaskLLM, Sparsity Pattern=2:42024.09 | 55.92 | — | |
| LLaMA-2 13B + MaskLLMBackbone=LLaMA-2 13B, Pruning Method=MaskLLM, Sparsity Pattern=2:42024.09 | 55.09 | — | |
| COMPOTBackbone=Llama2-7B, Training-free=true, CR=0.22026.02 | 51.8 | — | |
| Dobi-SVDBackbone=Llama2-7B, Training-free=false, CR=0.22026.02 | 51.5 | — | |
| LLaMA-2 7B + MaskLLMBackbone=LLaMA-2 7B, Pruning Method=MaskLLM, Sparsity Pattern=2:42024.09 | 50.91 | — | |
| PG pruningPrune Rate=30%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 50.34 | — | |
| SliceGPTPrune Rate=40%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 48.91 | — | |
| Swift-SVD*Ratio=0.8, Memory Footprint (GB)=10.1 GB, Zero-shot=true2026.04 | 48 | — | |
| BonsaiPrune Rate=30%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 47.05 | — | |
| Swift-SVDRatio=0.8, Memory Footprint (GB)=10.1 GB, Zero-shot=true2026.04 | 47 | — | |
| Wanda-spPrune Rate=30%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 46.62 | — | |
| Dobi-SVD(w)Ratio=0.8, Memory Footprint (GB)=10.1 GB, Zero-shot=true2026.04 | 46 | — | |
| SVD-LLM(W)Ratio=0.8, Memory Footprint (GB)=10.1 GB, Zero-shot=true2026.04 | 45 | — | |
| LLM-PrunerPrune Rate=30%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 43.64 | — | |
| GPT3 2B + MaskLLMBackbone=GPT3 2B, Pruning Method=MaskLLM, Sparsity Pattern=2:42024.09 | 42.64 | — | |
| PG pruningPrune Rate=40%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 42.44 | — | |
| ASVDRatio=0.8, Memory Footprint (GB)=10.1 GB, Zero-shot=true2026.04 | 41 | — | |
| BonsaiPrune Rate=40%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 40.63 | — | |
| SVD-LLM(W)Ratio=0.6, Memory Footprint (GB)=7.7 GB, Zero-shot=true2026.04 | 40 | — | |
| COMPOTBackbone=Llama2-7B, Training-free=true, CR=0.42026.02 | 39.6 | — | |
| Swift-SVD*Ratio=0.6, Memory Footprint (GB)=7.7 GB, Zero-shot=true2026.04 | 38 | — | |
| SliceGPTPrune Rate=50%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 37.96 | — | |
| Swift-SVDRatio=0.6, Memory Footprint (GB)=7.7 GB, Zero-shot=true2026.04 | 37 | — | |
| Dobi-SVD(w)Ratio=0.6, Memory Footprint (GB)=7.7 GB, Zero-shot=true2026.04 | 36 | — | |
| LLM-PrunerPrune Rate=40%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 35.76 | — | |
| Wanda-spPrune Rate=40%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 35.61 | — | |
| Dobi-SVDBackbone=Llama2-7B, Training-free=false, CR=0.42026.02 | 35.4 | — | |
| GPT3 843M + MaskLLMBackbone=GPT3 843M, Pruning Method=MaskLLM, Sparsity Pattern=2:42024.09 | 34.65 | — | |
| BonsaiPrune Rate=50%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 32.63 | — | |
| PG pruningPrune Rate=50%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 30.94 | — | |
| LLM-PrunerPrune Rate=50%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 29.74 | — | |
| Yat (pn+α)Parameterization=per-neuron, Scaling=learnable per-channel, Evaluation mode=zero-shot2026.05 | 29.62 | — | |
| Yat (sb+ca)Parameterization=shared (b, ε), Scaling=constant per-channel, Evaluation mode=zero-shot2026.05 | 29.62 | — | |
| Yat (sb+α)Parameterization=shared (b, ε), Scaling=learnable per-channel, Evaluation mode=zero-shot2026.05 | 29.52 | — | |
| Yat (pn+ca)Parameterization=per-neuron, Scaling=constant per-channel, Evaluation mode=zero-shot2026.05 | 29.2 | — | |
| GELUArchitecture=GELU MLP, Evaluation mode=zero-shot2026.05 | 29 | — | |
| COMPOTBackbone=Llama2-7B, Training-free=true, CR=0.62026.02 | 28.8 | — | |
| Dobi-SVDBackbone=Llama2-7B, Training-free=false, CR=0.62026.02 | 28.4 | — | |
| Swift-SVD*Ratio=0.4, Memory Footprint (GB)=5.3 GB, Zero-shot=true2026.04 | 28 | — | |
| Swift-SVDRatio=0.4, Memory Footprint (GB)=5.3 GB, Zero-shot=true2026.04 | 27 | — | |
| Wanda-spPrune Rate=50%, Backbone=LLaMA-2-7B, Evaluation Protocol=Zero-shot2024.06 | 26.81 | — | |
| Dobi-SVD(w)Ratio=0.4, Memory Footprint (GB)=5.3 GB, Zero-shot=true2026.04 | 24 | — | |
| SVD-LLM(W)Ratio=0.4, Memory Footprint (GB)=5.3 GB, Zero-shot=true2026.04 | 10 | — | |
| ASVDRatio=0.4, Memory Footprint (GB)=5.3 GB, Zero-shot=true2026.04 | 9 | — | |
| FWSVDRatio=0.8, Memory Footprint (GB)=10.1 GB, Zero-shot=true2026.04 | 8 | — | |
| ASVDRatio=0.6, Memory Footprint (GB)=7.7 GB, Zero-shot=true2026.04 | 8 | — | |
| FWSVDRatio=0.4, Memory Footprint (GB)=5.3 GB, Zero-shot=true2026.04 | 1 | — | |
| FWSVDRatio=0.6, Memory Footprint (GB)=7.7 GB, Zero-shot=true2026.04 | 0 | — | |
| AtomModel=LLaMA-2-7B, Precision=W4A4, Sequence Length=2048, Group Size=1282025.07 | — | 69.37 | |
| AtomModel=LLaMA-2-7B, Precision=W4A3, Sequence Length=2048, Group Size=1282025.07 | — | 62.54 | |
| AtomModel=LLaMA-3-8B, Precision=W4A4, Sequence Length=2048, Group Size=1282025.07 | — | 69.75 | |
| AtomModel=LLaMA-3-8B, Precision=W4A3, Sequence Length=2048, Group Size=1282025.07 | — | 67.78 | |
| AtomModel=Mistral, Precision=W4A4, Sequence Length=2048, Group Size=1282025.07 | — | 77.52 | |
| AtomModel=Mistral, Precision=W4A3, Sequence Length=2048, Group Size=1282025.07 | — | 73.24 | |
| EQAT-g64Backbone=Llama-2-7B, Bits=2.252026.06 | — | 51.58 | |
| EQAT-g64Backbone=Llama-2-13B, Bits=2.252026.06 | — | 55.27 | |
| EQAT-g64Backbone=Llama-2-70B, Bits=2.262026.06 | — | 61.78 | |
| EQAT-g64Backbone=Llama-3-8B, Bits=2.252026.06 | — | 51.86 | |
| EQAT-g64Backbone=Llama-3-70B, Bits=2.252026.06 | — | 61.6 | |
| FP16Model=LLaMA-2-7B, Precision=FP16, Sequence Length=20482025.07 | — | 75.95 | |
| FP16Model=LLaMA-3-8B, Precision=FP16, Sequence Length=20482025.07 | — | 79.61 | |
| FP16Model=Mistral, Precision=FP16, Sequence Length=20482025.07 | — | 81.18 | |
| FP16Backbone=Llama-2-7B, Bits=FP162026.06 | — | 57.16 | |
| FP16Backbone=Llama-2-13B, Bits=FP162026.06 | — | 60.07 | |
| FP16Backbone=Llama-2-70B, Bits=FP162026.06 | — | 64.77 | |
| FP16Backbone=Llama-3-8B, Bits=FP162026.06 | — | 60.17 | |
| FP16Backbone=Llama-3-70B, Bits=FP162026.06 | — | 66.36 | |
| LiftQuantBackbone=Llama-2-7B, Bits=2.022026.06 | — | 53.05 | |
| LiftQuantBackbone=Llama-2-7B, Bits=2.422026.06 | — | 55.1 | |
| LiftQuantBackbone=Llama-2-13B, Bits=2.022026.06 | — | 57.09 | |
| LiftQuantBackbone=Llama-2-13B, Bits=2.422026.06 | — | 58.64 | |
| LiftQuantBackbone=Llama-2-70B, Bits=2.012026.06 | — | 63.08 | |
| LiftQuantBackbone=Llama-2-70B, Bits=2.412026.06 | — | 64.31 | |
| LiftQuantBackbone=Llama-3-8B, Bits=2.022026.06 | — | 53.87 | |
| LiftQuantBackbone=Llama-3-70B, Bits=2.012026.06 | — | 62.31 | |
| OASISModel=LLaMA-2-7B, Precision=W4A4, Sequence Length=20482025.07 | — | 74.32 | |
| OASISModel=LLaMA-2-7B, Precision=W4A3, Sequence Length=20482025.07 | — | 68.28 | |
| OASISModel=LLaMA-3-8B, Precision=W4A4, Sequence Length=20482025.07 | — | 77 | |
| OASISModel=LLaMA-3-8B, Precision=W4A3, Sequence Length=20482025.07 | — | 73.8 | |
| OASISModel=Mistral, Precision=W4A4, Sequence Length=20482025.07 | — | 80.15 | |
| OASISModel=Mistral, Precision=W4A3, Sequence Length=20482025.07 | — | 78.58 | |
| OASIS-SModel=LLaMA-2-7B, Precision=W4A4, Sequence Length=20482025.07 | — | 72.57 | |
| OASIS-SModel=LLaMA-2-7B, Precision=W4A3, Sequence Length=20482025.07 | — | 67.58 | |
| OASIS-SModel=LLaMA-3-8B, Precision=W4A4, Sequence Length=20482025.07 | — | 75.88 | |
| OASIS-SModel=LLaMA-3-8B, Precision=W4A3, Sequence Length=20482025.07 | — | 71.98 | |
| OASIS-SModel=Mistral, Precision=W4A4, Sequence Length=20482025.07 | — | 78.99 | |
| OASIS-SModel=Mistral, Precision=W4A3, Sequence Length=20482025.07 | — | 76.02 | |
| QTIPBackbone=Llama-2-7B, Bits=2.002026.06 | — | 53.68 | |
| QTIPBackbone=Llama-2-13B, Bits=2.002026.06 | — | 58.13 | |
| QTIPBackbone=Llama-2-70B, Bits=2.002026.06 | — | 63.11 | |
| QuaRotModel=LLaMA-2-7B, Precision=W4A4, Sequence Length=20482025.07 | — | 71.63 | |
| QuaRotModel=LLaMA-2-7B, Precision=W4A3, Sequence Length=20482025.07 | — | 28.75 | |
| QuaRotModel=LLaMA-3-8B, Precision=W4A4, Sequence Length=20482025.07 | — | 61.73 | |
| QuaRotModel=LLaMA-3-8B, Precision=W4A3, Sequence Length=20482025.07 | — | 26.09 | |
| QuaRotModel=Mistral, Precision=W4A4, Sequence Length=20482025.07 | — | 72.88 |