Zero-shot Reasoning on Reasoning Suite (PIQA, HellaSwag, WinoGrande, ARC-e, ARC-c) (val test)
76.55Average AccuracyFP16
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| FP16Model=LLaMA2-70B, #Bits (W)=16, #Bits (A)=162026.05 | 76.55 | — | — | — | — | — | |
| FP16Model=Qwen3-32B, #Bits (W)=16, #Bits (A)=162026.05 | 76.33 | — | — | — | — | — | |
| DenseModel=LLaMA3-8B, PR=0%2025.05 | 75.62 | 85.56 | 79.27 | 77.94 | 78.84 | 56.49 | |
| FP16Model=Qwen3-14B, #Bits (W)=16, #Bits (A)=162026.05 | 75 | — | — | — | — | — | |
| FP16Backbone Model=LLaMA3.1 8B-Instruct2026.05 | 74.48 | 81.01 | 79.3 | 74.43 | 82.87 | 54.78 | |
| BWLAModel=LLaMA2-70B, #Bits (W)=1.16, #Bits (A)=162026.05 | 73.78 | — | — | — | — | — | |
| FP16Model=LLaMA3-8B, #Bits (W)=16, #Bits (A)=162026.05 | 72.67 | — | — | — | — | — | |
| SOARBackbone Model=LLaMA3.1 8B-Instruct2026.05 | 72.57 | 79.71 | 77.86 | 72.3 | 78.87 | 54.1 | |
| DensePruneRate=0%, Backbone=Mistral-7B-Instruct-v0.32024.06 | 72.54 | 81.77 | 64.84 | 74.51 | 84.22 | 57.34 | |
| RaZeRBackbone Model=LLaMA3.1 8B-Instruct2026.05 | 72.53 | 80.36 | 78.13 | 71.51 | 79.76 | 52.9 | |
| BWLAModel=LLaMA2-70B, #Bits (W)=1.16, #Bits (A)=62026.05 | 72.38 | — | — | — | — | — | |
| DenseModel=Phi-2, PR=0%2025.05 | 72.24 | 79.11 | 73.83 | 75.77 | 78.32 | 54.18 | |
| 4over6Backbone Model=LLaMA3.1 8B-Instruct2026.05 | 72.09 | 80.25 | 77.96 | 71.03 | 78.91 | 52.3 | |
| NVFP4Backbone Model=LLaMA3.1 8B-Instruct2026.05 | 71.91 | 78.84 | 77.98 | 71.82 | 78.7 | 52.22 | |
| DenseBase Model=Llama2-13b, Compression Ratio (%)=0, Evaluation Mode=Zero-shot2025.06 | 71.79 | 80.52 | 79.36 | 72.3 | 77.53 | 49.23 | |
| DenseModel=Llama2-13b, Evaluation protocol=Zero-shot2025.06 | 71.79 | 80.52 | 79.36 | 72.3 | 77.53 | 49.23 | |
| DenseBackbone=LLaMA-2 13B, Sparsity=0%2026.02 | 71.76 | 80.47 | 79.39 | 72.22 | 77.48 | 49.23 | |
| DenseModel=LLaMA2-13B, PR=0%2025.05 | 71.76 | 80.47 | 79.39 | 72.22 | 77.48 | 49.23 | |
| FP16Model=LLaMA2-13B, #Bits (W)=16, #Bits (A)=162026.05 | 71.71 | — | — | — | — | — | |
| FP16Backbone Model=Qwen3 8B2026.05 | 71.53 | 77.58 | 74.86 | 67.64 | 80.89 | 56.66 | |
| FP16Model=Qwen3-8B, #Bits (W)=16, #Bits (A)=162026.05 | 71.47 | — | — | — | — | — | |
| PocketLLMCompress Ratio=∼ 8×, Avg bits=3.982025.11 | 71.3 | 80.14 | 60.64 | 74.9 | 84.85 | 55.97 | |
| Qwen 3-14BCompress Ratio=–, Avg bits=322025.11 | 71.23 | 80.58 | 61.86 | 74.35 | 83.54 | 55.8 | |
| SOARBackbone Model=Qwen3 8B2026.05 | 70.68 | 77.26 | 73.19 | 68.35 | 79.42 | 55.2 | |
| PocketLLMCompress Ratio=∼ 10×, Avg bits=3.022025.11 | 70.19 | 79.49 | 57.72 | 74.2 | 84.22 | 55.25 | |
| RaZeRBackbone Model=Qwen3 8B2026.05 | 70.12 | 76.22 | 73.58 | 67.72 | 78.41 | 54.69 | |
| GPTQCompress Ratio=∼ 8×, Avg bits=4.002025.11 | 69.8 | 79.7 | 60.6 | 73.2 | 81.5 | 54 | |
| AWQCompress Ratio=∼ 8×, Avg bits=4.002025.11 | 69.72 | 79.9 | 60.6 | 71.7 | 82.2 | 54.2 | |
| 4over6Backbone Model=Qwen3 8B2026.05 | 69.15 | 75.68 | 73.24 | 64.09 | 79.25 | 53.5 | |
| DenseBackbone=LLaMA-2 7B, Sparsity=0%2026.02 | 69 | 79.11 | 75.99 | 69.06 | 74.58 | 46.25 | |
| LLaMA-2 7B (original)Sparsity=0%, Calibration Dataset=None2026.02 | 69 | 79.11 | 75.99 | 69.06 | 74.58 | 46.25 | |
| DenseModel=LLaMA2-7B, PR=0%2025.05 | 69 | 79.11 | 75.99 | 69.06 | 74.58 | 46.25 | |
| FP16Model=LLaMA2-7B, #Bits (W)=16, #Bits (A)=162026.05 | 68.96 | — | — | — | — | — | |
| DenseModel=Llama2-7b, Evaluation protocol=Zero-shot2025.06 | 68.95 | 79.11 | 75.99 | 68.82 | 74.58 | 46.25 | |
| NVFP4Backbone Model=Qwen3 8B2026.05 | 68.75 | 75.63 | 73.27 | 66.54 | 73.27 | 55.03 | |
| DenseSparsity=0%, Model=Llama-3-8B, Evaluation Protocol=Zero-shot Accuracy2025.05 | 68.6 | 79.7 | 60.2 | 72.6 | 80.1 | 50.4 | |
| DensePruning Rate=0%, Backbone=LLaMA-3-8B, Weight fine-tuning=Alpaca dataset2024.06 | 68.59 | 79.71 | 60.19 | 72.61 | 80.09 | 50.34 | |
| TRSP -ℓ2Model=LLaMA3-8B, PR=25%2025.05 | 68.44 | 77.25 | 72.26 | 71.63 | 71.49 | 49.58 | |
| TRSP -ℓ1Model=LLaMA3-8B, PR=25%2025.05 | 68.38 | 77.36 | 72.82 | 71.33 | 70.75 | 49.66 | |
| FP16Backbone Model=Qwen3 4B2026.05 | 68.38 | 75.08 | 68.46 | 65.9 | 78.45 | 54.01 | |
| ProcrustesGPT (Kron)Base Model=Llama2-13b, Compression Ratio (%)=14.30, Evaluation Mode=Zero-shot, Factorization=Kronecker products2025.06 | 68.37 | 77.58 | 74.29 | 70.24 | 74.16 | 45.56 | |
| BWLAModel=Qwen3-32B, #Bits (W)=1.16, #Bits (A)=162026.05 | 68.29 | — | — | — | — | — | |
| SLEBBase Model=Llama2-13b, Compression Ratio (%)=12.19, Evaluation Mode=Zero-shot2025.06 | 68.25 | 78.18 | 74.11 | 69.85 | 72.77 | 46.33 | |
| ProcrustesGPTModel=Llama2-13b, Struct.=Kron., Coef.=sqrt(D+1), Evaluation protocol=Zero-shot, Norm=Weighted2025.06 | 68.07 | 77.53 | 73.94 | 70.88 | 73.23 | 44.8 | |
| ProcrustesGPTModel=Llama2-13b, Struct.=Kron., Coef.=log(D+1), Evaluation protocol=Zero-shot, Norm=Weighted2025.06 | 67.93 | 77.58 | 73.43 | 70.48 | 73.19 | 44.97 | |
| DISP-LLMBase Model=Llama2-13b, Compression Ratio (%)=14.60, Evaluation Mode=Zero-shot2025.06 | 67.81 | 76.93 | 74.77 | 69.61 | 70.12 | 47.61 | |
| ProcrustesGPT (GS)Base Model=Llama2-13b, Compression Ratio (%)=14.30, Evaluation Mode=Zero-shot, Factorization=GS matrices2025.06 | 67.5 | 76.77 | 71.71 | 70.56 | 73.4 | 45.05 | |
| FlattenGPTBackbone=LLaMA-2 13B, Sparsity=24.37%2026.02 | 67.5 | 76.33 | 73.44 | 71.11 | 72.1 | 44.54 | |
| BWLAModel=Qwen3-32B, #Bits (W)=1.16, #Bits (A)=62026.05 | 67.17 | — | — | — | — | — | |
| ProcrustesGPTModel=Llama2-13b, Struct.=GS, Coef.=sqrt(D+1), Evaluation protocol=Zero-shot, Norm=Weighted2025.06 | 67.16 | 76.5 | 71.19 | 70.32 | 73.32 | 44.45 | |
| FP16Backbone Model=LLaMA3.2 3B-Instruct2026.05 | 66.83 | 76.66 | 71.55 | 66.83 | 71.09 | 46.25 | |
| DenseBackbone=Baichuan-2 7B, Sparsity=0%2026.02 | 66.73 | 77.48 | 72.18 | 68.27 | 72.98 | 42.75 | |
| ARB-LLMModel=Qwen3-32B, #Bits (W)=1.07, #Bits (A)=162026.05 | 66.51 | — | — | — | — | — | |
| FlattenGPT†Sparsity=20%, Calibration Dataset=WikiText-2, Fine-tuned on Alpaca=true2026.02 | 66.24 | 74.97 | 73.01 | 68.75 | 67.4 | 45.05 | |
| ShortGPTModel=LLaMA3-8B, PR=25%2025.05 | 66.17 | 75.38 | 70.12 | 69.25 | 68.54 | 47.57 | |
| SOARBackbone Model=LLaMA3.2 3B-Instruct2026.05 | 66 | 76.17 | 70.7 | 67.88 | 70.03 | 45.22 | |
| SOARBackbone Model=Qwen3 4B2026.05 | 65.86 | 74.32 | 66.68 | 63.14 | 74.24 | 50.94 | |
| RaZeRBackbone Model=Qwen3 4B2026.05 | 65.79 | 73.61 | 66.54 | 63.77 | 74.79 | 50.26 | |
| LLM SurgeonSparsity=20%, Calibration Dataset=WikiText-2, Fine-tuned on Alpaca=false2026.02 | 65.59 | 77.09 | 71.3 | 66.3 | 71.36 | 41.89 | |
| ModeGPTSparsity=20%, Calibration Dataset=Alpaca, Fine-tuned on Alpaca=false2026.02 | 65.52 | 76.22 | 69.59 | 68.19 | 71.71 | 41.89 | |
| DenseBackbone=Qwen-1.5 7B, Sparsity=0%2026.02 | 65.48 | 79.22 | 76.92 | 66.46 | 62.16 | 42.66 | |
| RaZeRBackbone Model=LLaMA3.2 3B-Instruct2026.05 | 65.19 | 76.17 | 70.79 | 67.01 | 67.85 | 44.11 | |
| TRSP -ℓ1Model=LLaMA2-13B, PR=25%2025.05 | 65.18 | 75.06 | 64.96 | 69.18 | 71.43 | 45.26 | |
| TRSP -ℓ2Model=LLaMA2-13B, PR=25%2025.05 | 65.11 | 74.56 | 64.79 | 69.34 | 71.25 | 45.63 | |
| NVFP4Backbone Model=Qwen3 4B2026.05 | 65.09 | 73.07 | 65.94 | 61.01 | 74.49 | 50.94 | |
| NVFP4Backbone Model=LLaMA3.2 3B-Instruct2026.05 | 65.02 | 75.63 | 70.38 | 65.11 | 69.36 | 44.62 | |
| ModeGPT†Sparsity=20%, Calibration Dataset=Alpaca, Fine-tuned on Alpaca=true2026.02 | 64.99 | 77.2 | 68.07 | 66.3 | 70.45 | 42.92 | |
| LaCoModel=LLaMA3-8B, PR=25%2025.05 | 64.86 | 74.69 | 66.36 | 67.52 | 69.43 | 46.31 | |
| 4over6Backbone Model=LLaMA3.2 3B-Instruct2026.05 | 64.74 | 75.63 | 70.03 | 65.43 | 69.28 | 43.34 | |
| DensePrune Rate=0%, Model backbone=LLaMA-2-7B, Evaluation protocol=Zero-shot, Weight fine-tuning=true, Fine-tuning dataset=Alpaca2024.06 | 64.69 | 78.02 | 57.17 | 68.43 | 76.3 | 43.51 | |
| EvoPressSparsity=50%, Latency=139m, Model=Llama-3-8B, Evaluation Protocol=Zero-shot Accuracy2025.05 | 64.67 | 76.17 | 54.92 | 72.14 | 76.18 | 43.94 | |
| ProcrustesGPTModel=Llama2-7b, Struct.=Kron., Coef.=sqrt(D+1), Evaluation protocol=Zero-shot, Norm=Weighted2025.06 | 64.64 | 76.39 | 70.2 | 66.3 | 68.01 | 42.32 | |
| BlockPrunerBackbone=LLaMA-2 13B, Sparsity=25.12%2026.02 | 64.53 | 76.93 | 72.2 | 66.3 | 65.82 | 41.38 | |
| RTNCompress Ratio=∼ 8×, Avg bits=4.002025.11 | 64.52 | 76.3 | 53.5 | 65.4 | 76.4 | 51 | |
| UniCuCoSparsity=50%, Latency=<1s, Model=Llama-3-8B, Evaluation Protocol=Zero-shot Accuracy2025.05 | 64.32 | 76.28 | 54.3 | 71.27 | 75.84 | 43.9 | |
| OWLSparsity=50%, Latency=30m, Model=Llama-3-8B, Evaluation Protocol=Zero-shot Accuracy2025.05 | 64.3 | 75.7 | 54 | 72.2 | 75.8 | 43.8 | |
| ProcrustesGPTModel=Llama2-7b, Struct.=Kron., Coef.=log(D+1), Evaluation protocol=Zero-shot, Norm=Weighted2025.06 | 64.28 | 73.94 | 69.72 | 67.4 | 68.35 | 41.98 | |
| UniformSparsity=50%, Latency=<1s, Model=Llama-3-8B, Evaluation Protocol=Zero-shot Accuracy2025.05 | 64.26 | 76.1 | 54.2 | 71.7 | 75.7 | 43.6 | |
| ProcrustesGPT (GS)Base Model=Llama2-13b, Compression Ratio (%)=25.48, Evaluation Mode=Zero-shot, Factorization=GS matrices2025.06 | 64.1 | 74.71 | 66.12 | 67.17 | 71.38 | 41.04 | |
| Shortened LLaMAModel=LLaMA3-8B, PR=25%2025.05 | 64.09 | 73.62 | 68.55 | 68.73 | 66.16 | 43.39 | |
| FlattenGPTSparsity=20%, Calibration Dataset=WikiText-2, Fine-tuned on Alpaca=false2026.02 | 63.83 | 74.59 | 70.74 | 67.4 | 64.44 | 41.98 | |
| 4over6Backbone Model=Qwen3 4B2026.05 | 63.83 | 71.65 | 66.11 | 60.3 | 73.06 | 48.04 | |
| LLaMA3.2Model Size=3B, Bit-width=16, Evaluation Protocol=zero-shot2026.02 | 63.6 | 76.8 | 55.2 | 69.1 | 74.5 | 42.2 | |
| ARB-LLMModel=LLaMA2-70B, #Bits (W)=1.07, #Bits (A)=162026.05 | 63.36 | — | — | — | — | — | |
| ProcrustesGPT (Kron)Base Model=Llama2-13b, Compression Ratio (%)=25.48, Evaluation Mode=Zero-shot, Factorization=Kronecker products2025.06 | 63.2 | 75.03 | 66.42 | 66.06 | 70.03 | 38.48 | |
| SLEBModel=LLaMA3-8B, PR=25%2025.05 | 63.12 | 72.74 | 67.74 | 64.12 | 65.84 | 45.16 | |
| RMBackbone=LLaMA-2 13B, Sparsity=24.37%2026.02 | 63.05 | 73.72 | 66.8 | 66.61 | 66.12 | 41.98 | |
| ShortGPTModel=LLaMA2-13B, PR=25%2025.05 | 62.96 | 73.26 | 62.25 | 68.37 | 67.54 | 43.38 | |
| BWLAModel=Qwen3-14B, #Bits (W)=1.16, #Bits (A)=162026.05 | 62.65 | — | — | — | — | — | |
| ShortGPTBackbone=LLaMA-2 13B, Sparsity=24.37%2026.02 | 62.6 | 72.74 | 67.8 | 70.8 | 60.35 | 41.3 | |
| FlattenGPTBackbone=LLaMA-2 7B, Sparsity=21.02%2026.02 | 62.49 | 72.74 | 68.45 | 66.54 | 63.43 | 41.3 | |
| AWQCompress Ratio=∼ 10×, Avg bits=3.002025.11 | 62.44 | 76 | 55.9 | 62.7 | 73 | 44.6 | |
| BWLAModel=LLaMA2-13B, #Bits (W)=1.16, #Bits (A)=162026.05 | 62.38 | — | — | — | — | — | |
| SVD-LLMBase Model=Llama2-13b, Compression Ratio (%)=14.64, Evaluation Mode=Zero-shot2025.06 | 62.19 | 73.83 | 63.92 | 68.35 | 65.61 | 39.25 | |
| LLM-Pruner†Sparsity=18.82%, Calibration Dataset=WikiText-2, Fine-tuned on Alpaca=true2026.02 | 62.15 | 77.48 | 67.13 | 61.88 | 65.78 | 38.48 | |
| DenseModel=OPT-13b, Evaluation protocol=Zero-shot2025.06 | 61.89 | 76.82 | 69.88 | 65.19 | 61.83 | 35.75 | |
| DenseModel=OPT-13B, PR=0%2025.05 | 61.79 | 76.82 | 69.81 | 64.8 | 61.87 | 35.67 | |
| ProcrustesGPTModel=OPT-13b, Struct.=Kron., Coef.=sqrt(D+1), Evaluation protocol=Zero-shot, Norm=Weighted2025.06 | 61.73 | 76.44 | 68.41 | 64.96 | 62.25 | 36.6 | |
| DISP-LLMBase Model=Llama2-13b, Compression Ratio (%)=25.36, Evaluation Mode=Zero-shot2025.06 | 61.67 | 73.56 | 69.4 | 63.3 | 61.83 | 40.27 | |
| ProcrustesGPTModel=Llama2-7b, Struct.=GS, Coef.=sqrt(D+1), Evaluation protocol=Zero-shot, Norm=Weighted2025.06 | 61.65 | 74.37 | 66.91 | 67.4 | 62.88 | 36.69 |