Language Modeling on WikiText
0.2838PPLLlama 3.1-70B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Llama 3.1-70BPrecision=BF162026.02 | 0.2838 | — | |
| Attn-QATModel=Llama 3.1-70B, Precision=Attn-QAT2026.02 | 0.3076 | — | |
| Llama 3.1-70BPrecision=FP42026.02 | 0.3275 | — | |
| Qwen3-14BPrecision=BF162026.02 | 0.57 | — | |
| Qwen3-14BPrecision=FP42026.02 | 0.5763 | — | |
| Attn-QATModel=Qwen3-14B, Precision=Attn-QAT2026.02 | 0.5778 | — | |
| LLaMA-3.2-3B-HFTput (4k/1k)=4311, n_layers=28, d_model=3072, r_mlp/attn=3, GQA=3, N_non-embed=2.82B2025.10 | 1.5164 | — | |
| Qwen2.5-3B-HFTput (4k/1k)=6470, n_layers=36, d_model=2048, r_mlp/attn=7.167, GQA=8, N_non-embed=2.77B2025.10 | 1.6185 | — | |
| Panda-3BTput (4k/1k)=3335, n_layers=28, d_model=4096, r_mlp/attn=1, GQA=3, N_non-embed=2.82B2025.10 | 1.6454 | — | |
| Surefire-3BTput (4k/1k)=4842, n_layers=28, d_model=4096, r_mlp/attn=1, GQA=7, N_non-embed=2.82B2025.10 | 1.6462 | — | |
| LLaMA-3.2-3BTput (4k/1k)=4311, n_layers=28, d_model=3072, r_mlp/attn=3, GQA=3, N_non-embed=2.82B2025.10 | 1.6489 | — | |
| BF16Model=Llama-3 70B, Setting=W16A162025.12 | 2.92 | — | |
| Dense (Baseline)Pruning Ratio=0%, Model=Llama2-70B2026.02 | 3.12 | — | |
| SpQRModel=Llama-3 70B, Setting=W4A162025.12 | 3.2 | — | |
| SQ-formatModel=Llama-3 70B, Setting=W4A(SQ6)2025.12 | 3.31 | — | |
| DenseModel=LLaMA2-70B, Sparsity rate=0%, Pruning type=none2026.03 | 3.32 | — | |
| DeepSeek-R1 (BF16)Model=DeepSeek-R1, Setting=W16A16, Sparsity=02025.12 | 3.33 | — | |
| SQ-formatModel=DeepSeek-R1, Setting=W(SQ5)A8, Sparsity=0.8752025.12 | 3.39 | — | |
| SQ-formatModel=Llama-3 70B, Setting=W4A(SQ5)2025.12 | 3.45 | — | |
| GPTQModel=Llama-3 70B, Setting=W4A82025.12 | 3.48 | — | |
| PPPruning Ratio=20%, Online=true, Model=Llama2-70B2026.02 | 3.61 | — | |
| SpinQuantModel=Llama-3 70B, Setting=W4A82025.12 | 3.73 | — | |
| SQ-formatModel=Llama-3 70B, Setting=W(SQ6)A42025.12 | 3.79 | — | |
| POPPruning Ratio=20%, Online=true, Model=Llama2-70B2026.02 | 3.92 | — | |
| TýrPruning Ratio=20%, Online=false, Model=Llama2-70B2026.02 | 4.25 | — | |
| SQ-formatModel=Llama-3 70B, Setting=W(SQ5)A42025.12 | 4.49 | — | |
| Dense (Baseline)Pruning Ratio=0%, Model=Llama2-13B2026.02 | 4.57 | — | |
| GPTQModel=Llama-3 70B, Setting=W4A42025.12 | 4.58 | — | |
| PPPruning Ratio=40%, Online=true, Model=Llama2-70B2026.02 | 4.62 | — | |
| SQ-formatModel=Llama-3 70B, Setting=W(SQ4.5)A42025.12 | 4.67 | — | |
| DenseModel=LLaMA2-13B, Sparsity rate=0%, Pruning type=none2026.03 | 4.88 | — | |
| Dense (Baseline)Pruning Ratio=0%, Model=Llama2-7B2026.02 | 5.12 | — | |
| DenseAvg. bits=16, Backbone=Llama2-7B2025.05 | 5.12 | — | |
| PPPruning Ratio=20%, Online=true, Model=Llama2-13B2026.02 | 5.25 | — | |
| TýrPruning Ratio=20%, Online=false, Model=Llama2-13B2026.02 | 5.41 | — | |
| FPBackbone=Llama 2 7B, Weight quantization=None2026.03 | 5.47 | — | |
| DenseModel=LLaMA2-7B, Sparsity rate=0%, Pruning type=none2026.03 | 5.47 | — | |
| DenseAvg. bits=16, PV-tuning=false2025.05 | 5.54 | — | |
| DenseAvg. bits=162025.05 | 5.54 | — | |
| Dense (Baseline)Pruning Ratio=0%, Model=Qwen2-57B-A14B2026.02 | 5.56 | — | |
| POPPruning Ratio=40%, Online=true, Model=Llama2-70B2026.02 | 5.63 | — | |
| TýrPruning Ratio=40%, Online=false, Model=Llama2-70B2026.02 | 5.64 | — | |
| POPPruning Ratio=20%, Online=true, Model=Qwen2-57B-A14B2026.02 | 5.75 | — | |
| AQLM + PVAvg. bits=2.3, Backbone=Llama2-7B2025.05 | 5.84 | — | |
| CalderaAvg. bits=2.4, Backbone=Llama2-7B2025.05 | 5.84 | — | |
| DBF + PVAvg. bits=2.3, Backbone=Llama2-7B2025.05 | 5.85 | — | |
| QTIPAvg. bits=2, Backbone=Llama2-7B2025.05 | 5.86 | — | |
| DBFAvg. bits=2.3, Backbone=Llama2-7B2025.05 | 5.87 | — | |
| POPPruning Ratio=20%, Online=true, Model=Llama2-13B2026.02 | 5.94 | — | |
| GPTQ FlatQuantBackbone=Llama 2 7B, Weight quantization=GPTQ, Transform method=FlatQuant, Train=true2026.03 | 5.94 | — | |
| CAT (block)Backbone=Llama 2 7B, Weight quantization=RTN, Transform method=CAT (block), Train=true2026.03 | 5.95 | — | |
| CAT (block)Backbone=Llama 2 7B, Weight quantization=GPTQ, Transform method=CAT (block), Train=true2026.03 | 5.97 | — | |
| CAT (block)Backbone=Llama 2 7B, Weight quantization=GPTQ, Transform method=CAT (block), Train=false2026.03 | 6.01 | — | |
| FLAPPruning Ratio=20%, Online=false, Model=Qwen2-57B-A14B2026.02 | 6.02 | — | |
| RTN FlatQuantBackbone=Llama 2 7B, Weight quantization=RTN, Transform method=FlatQuant, Train=true2026.03 | 6.05 | — | |
| DBF + PVAvg. bits=2, Backbone=Llama2-7B2025.05 | 6.09 | — | |
| CAT (block)Backbone=Llama 2 7B, Weight quantization=RTN, Transform method=CAT (block), Train=false2026.03 | 6.11 | — | |
| DBFAvg. bits=2, Backbone=Llama2-7B2025.05 | 6.14 | — | |
| FPBackbone=Llama 3 8B, Weight quantization=None2026.03 | 6.14 | — | |
| PPPruning Ratio=20%, Online=true, Model=Llama2-7B2026.02 | 6.17 | — | |
| QUIP#Avg. bits=2, Backbone=Llama2-7B2025.05 | 6.19 | — | |
| POPPruning Ratio=40%, Online=true, Model=Qwen2-57B-A14B2026.02 | 6.21 | — | |
| GPTQ QuaRotBackbone=Llama 2 7B, Weight quantization=GPTQ, Transform method=QuaRot, Train=false2026.03 | 6.22 | — | |
| GPTQ SpinQuantBackbone=Llama 2 7B, Weight quantization=GPTQ, Transform method=SpinQuant, Train=true2026.03 | 6.26 | — | |
| CalderaAvg. bits=2.1, Backbone=Llama2-7B2025.05 | 6.3 | — | |
| TýrPruning Ratio=20%, Online=false, Model=Llama2-7B2026.02 | 6.44 | — | |
| FLAPPruning Ratio=20%, Online=false, Model=Llama2-13B2026.02 | 6.64 | — | |
| AQLMAvg. bits=2.3, PV-tuning=true2025.05 | 6.76 | — | |
| AQLM + PVAvg. bits=2.32025.05 | 6.76 | — | |
| Dense (Baseline)Pruning Ratio=0%, Model=Qwen1.5-MoE-A2.7B2026.02 | 6.79 | — | |
| POPPruning Ratio=20%, Online=true, Model=Llama2-7B2026.02 | 6.86 | — | |
| DBFAvg. bits=2.3, PV-tuning=true2025.05 | 6.86 | — | |
| DBF + PVAvg. bits=2.32025.05 | 6.86 | — | |
| RTN SpinQuantBackbone=Llama 2 7B, Weight quantization=RTN, Transform method=SpinQuant, Train=true2026.03 | 6.96 | — | |
| DBFAvg. bits=2.3, PV-tuning=false2025.05 | 6.97 | — | |
| FPBackbone=Ministral 8B it, Weight quantization=None2026.03 | 6.97 | — | |
| DBF + PVAvg. bits=1.5, Backbone=Llama2-7B2025.05 | 7.01 | — | |
| Dense (Baseline)Pruning Ratio=0%, Model=Qwen3-32B2026.02 | 7.02 | — | |
| DBFAvg. bits=1.5, Backbone=Llama2-7B2025.05 | 7.16 | — | |
| TýrPruning Ratio=20%, Online=false, Model=Qwen3-32B2026.02 | 7.2 | — | |
| GPTQ FlatQuantBackbone=Llama 3 8B, Weight quantization=GPTQ, Transform method=FlatQuant, Train=true2026.03 | 7.25 | — | |
| CAT (block)Backbone=Llama 3 8B, Weight quantization=GPTQ, Transform method=CAT (block), Train=true2026.03 | 7.25 | — | |
| BF16Model=Llama-3 8B, Setting=W16A162025.12 | 7.26 | — | |
| Llama3 8BCompression Ratio=Baseline2025.09 | 7.3 | — | |
| Llama3 8B2026.02 | 7.3 | — | |
| Llama3-8BBackbone=Llama3-8B, CR=None2026.02 | 7.3 | — | |
| DBFAvg. bits=2, PV-tuning=true2025.05 | 7.3 | — | |
| DBF + PVAvg. bits=22025.05 | 7.3 | — | |
| AQLM + PVAvg. bits=1.5, Backbone=Llama2-7B2025.05 | 7.32 | — | |
| Llama 3.1 8BAttn CR=N/A2025.08 | 7.33 | — | |
| QTIPAvg. bits=2, PV-tuning=false2025.05 | 7.33 | — | |
| QTIPAvg. bits=22025.05 | 7.33 | — | |
| CAT (block)Backbone=Llama 3 8B, Weight quantization=RTN, Transform method=CAT (block), Train=true2026.03 | 7.33 | — | |
| PPPruning Ratio=40%, Online=true, Model=Llama2-13B2026.02 | 7.34 | — | |
| CalderaAvg. bits=2.4, PV-tuning=false2025.05 | 7.34 | — | |
| DBFAvg. bits=2, PV-tuning=false2025.05 | 7.48 | — | |
| POPPruning Ratio=20%, Online=true, Model=Qwen1.5-MoE-A2.7B2026.02 | 7.51 | — | |
| FLAPPruning Ratio=20%, Online=false, Model=Llama2-7B2026.02 | 7.53 | — | |
| SpQRModel=Llama-3 8B, Setting=W4A162025.12 | 7.55 | — | |
| CAT (block)Backbone=Llama 3 8B, Weight quantization=GPTQ, Transform method=CAT (block), Train=false2026.03 | 7.6 | — |