Language Modeling on WikiText2 v1 (test)
1.7PerplexityMeta 'FP8'
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Meta 'FP8'Bits=16, Precision Detail=16 ATTN. / 8 MLP, Model=Llama 3.1 405B Instruct, Context Length=81922024.06 | 1.7 | — | — | — | |
| QTIPBits=4, Model=Llama 3.1 405B Instruct, Context Length=81922024.06 | 1.79 | — | — | — | |
| QTIPBits=3, Model=Llama 3.1 405B Instruct, Context Length=81922024.06 | 2.05 | — | — | — | |
| UnquantizedBackbone=LLaMa-2 70B, Avg Bits=16, Evaluation Protocol=Zero-shot2024.05 | 3.12 | — | — | — | |
| QTIPBits=2, Model=Llama 3.1 405B Instruct, Context Length=81922024.06 | 3.29 | — | — | — | |
| LLaMA-2 70BPruning Type=Original, Sparsity=0%2026.02 | 3.32 | — | — | — | |
| BF16Bits=16, Model=Llama 3.1 70B Instruct, Context Length=81922024.06 | 3.52 | — | — | — | |
| QuIPModel Variant=Llama-1 7B, Setting=w4, #Bit=42025.03 | 3.53 | — | — | — | |
| QTIPBits=4, Model=Llama 3.1 70B Instruct, Context Length=81922024.06 | 3.73 | — | — | — | |
| CALDERABackbone=LLaMa-2 70B, Rank=256, B_L (= B_R)=4, Avg Bits=2.2, Quantization Bits (B_Q)=2, Evaluation Protocol=Zero-shot2024.05 | 3.98 | — | — | — | |
| CALDERABackbone=LLaMa-2 70B, Rank=128, B_L (= B_R)=4, Avg Bits=2.1, Quantization Bits (B_Q)=2, Evaluation Protocol=Zero-shot2024.05 | 4.11 | — | — | — | |
| QTIPBits=3, Model=Llama 3.1 70B Instruct, Context Length=81922024.06 | 4.12 | — | — | — | |
| QuIP# (No FT)Backbone=LLaMa-2 70B, Rank=0, B_L (= B_R)=0, Avg Bits=2, Quantization Bits (B_Q)=2, Evaluation Protocol=Zero-shot2024.05 | 4.16 | — | — | — | |
| GPTVQModel Variant=Llama-2 70B, Setting=w2g64, #Bit=2.252025.03 | 4.39 | — | — | — | |
| SliceGPTBase Model=LLaMA-2 70B, Pruning Type=Width, Sparsity=20%2026.02 | 4.44 | — | — | — | |
| UnquantizedBackbone=LLaMa-2 13B, Avg Bits=16, Evaluation Protocol=Zero-shot2024.05 | 4.57 | — | — | — | |
| FlattenGPTBase Model=LLaMA-2 70B, Pruning Type=Depth, Sparsity=19.84%2026.02 | 4.79 | — | — | — | |
| DenseBackbone=Mistral-7B, Sparsity=0%2025.05 | 4.82 | — | — | — | |
| ClusComp (optimized)Model Variant=Llama-2 70B, Setting=g9, #Bit=<2.012025.03 | 4.83 | — | — | 1.85 | |
| LLaMA-2 13BPruning Type=Original, Sparsity=0%2026.02 | 4.88 | — | — | — | |
| SLEBBase Model=LLaMA-2 70B, Pruning Type=Depth, Sparsity=19.84%2026.02 | 4.88 | — | — | — | |
| ShiftAddLLMBits=4, Backbone=LLaMA-2-13B, Group Size=1282024.06 | 4.96 | — | — | — | |
| AWQBits=4, Backbone=LLaMA-2-13B, Group Size=1282024.06 | 4.97 | — | — | — | |
| OPTQBits=4, Backbone=LLaMA-2-13B, Group Size=1282024.06 | 4.98 | — | — | — | |
| LUT-GEMMBits=4, Backbone=LLaMA-2-13B, Group Size=1282024.06 | 5.06 | — | — | — | |
| QTIPBits=2, Model=Llama 3.1 70B Instruct, Context Length=81922024.06 | 5.08 | — | — | — | |
| UnquantizedBackbone=LLaMa-2 7B, Avg Bits=16, Evaluation Protocol=Zero-shot2024.05 | 5.12 | — | — | — | |
| Llama 2-7BRatio=-, Avg bits=322025.11 | 5.12 | — | — | — | |
| QTIPRatio=∼ 8×, Avg bits=4.01, Weight Updation=Yes2025.11 | 5.17 | — | — | — | |
| AQLMRatio=∼ 8×, Avg bits=4.04, Weight Updation=Yes2025.11 | 5.21 | — | — | — | |
| QATW Bits=4, Backbone=LLaMA 13B, Group Size=None2023.05 | 5.26 | — | — | — | |
| PocketLLMRatio=∼ 8×, Avg bits=3.98, Weight Updation=Yes2025.11 | 5.27 | — | — | — | |
| SpQRRatio=∼ 8×, Avg bits=3.98, Weight Updation=Yes2025.11 | 5.28 | — | — | — | |
| QuIP#Ratio=∼ 8×, Avg bits=4.02, Weight Updation=Yes2025.11 | 5.29 | — | — | — | |
| PEQAW Bits=4, Backbone=LLaMA 13B, Group Size=None2023.05 | 5.3 | — | — | — | |
| LoRA + OPTQW Bits=4, Backbone=LLaMA 13B, LoRA configuration=QV4, Group Size=None2023.05 | 5.31 | — | — | — | |
| CALDERABackbone=LLaMa-2 13B, Rank=256, B_L (= B_R)=4, Avg Bits=2.32, Quantization Bits (B_Q)=2, Evaluation Protocol=Zero-shot2024.05 | 5.41 | — | — | — | |
| LLaMA-2 7BPruning Type=Original, Sparsity=0%2026.02 | 5.47 | — | — | — | |
| WandaBase Model=LLaMA-2 70B, Pruning Type=Width, Sparsity=2:4 (50%)2026.02 | 5.48 | — | — | — | |
| GPTQRatio=∼ 8×, Avg bits=4.00, Weight Updation=Yes2025.11 | 5.49 | — | — | — | |
| UnquantizedBackbone=LLaMa-3 8B, Avg Bits=16, Evaluation Protocol=Zero-shot2024.05 | 5.54 | — | — | — | |
| PEQAW Bits=3, Backbone=LLaMA 13B, Group Size=None2023.05 | 5.54 | — | — | — | |
| DenseBackbone=Llama-3-8B, Sparsity=0%2025.05 | 5.54 | — | — | — | |
| ShiftAddLLMBits=4, Backbone=LLaMA-2-7B, Group Size=1282024.06 | 5.58 | — | — | — | |
| QATW Bits=3, Backbone=LLaMA 13B, Group Size=None2023.05 | 5.59 | — | — | — | |
| AWQBits=4, Backbone=LLaMA-2-7B, Group Size=1282024.06 | 5.6 | — | — | — | |
| VPTQRatio=∼ 8×, Avg bits=4.01, Weight Updation=Yes2025.11 | 5.64 | — | — | — | |
| FP16 (Baseline)Model Variant=Llama-1 7B, Setting=-, #Bit=162025.03 | 5.68 | — | — | — | |
| OPTQBits=4, Backbone=LLaMA-2-7B, Group Size=1282024.06 | 5.69 | — | — | — | |
| PV-TuningBits=2.01, Model=Llama 3.1 70B Instruct, Context Length=81922024.06 | 5.7 | — | — | — | |
| SpareGPTBase Model=LLaMA-2 70B, Pruning Type=Width, Sparsity=2:4 (50%)2026.02 | 5.7 | — | — | — | |
| CALDERABackbone=LLaMa-2 13B, Rank=128, B_L (= B_R)=4, Avg Bits=2.16, Quantization Bits (B_Q)=2, Evaluation Protocol=Zero-shot2024.05 | 5.72 | — | — | — | |
| ClusComp (optimized)Model Variant=Llama-1 7B, Setting=g4n65500, #Bit=≤4.142025.03 | 5.73 | — | — | 4.14 | |
| EvoPressBackbone=Mistral-7B, Sparsity=12.5%, Latency=26.7m2025.05 | 5.74 | — | — | — | |
| PocketLLMRatio=∼ 8×, Avg bits=3.98, Weight Updation=No2025.11 | 5.74 | — | — | — | |
| QATW Bits=4, Backbone=LLAMA 7B, Group Size=None2023.05 | 5.76 | — | — | — | |
| ShiftAddLLMBits=4, Backbone=LLaMA-1-7B, Group Size=1282024.06 | 5.76 | — | — | — | |
| AffineQuantModel Variant=Llama-1 7B, Setting=w4g128, #Bit=4.132025.03 | 5.77 | — | — | — | |
| OmniQuantModel Variant=Llama-1 7B, Setting=w4g128, #Bit=4.162025.03 | 5.77 | — | — | — | |
| LUT-GEMMBits=4, Backbone=LLaMA-2-7B, Group Size=1282024.06 | 5.78 | — | — | — | |
| AWQBits=4, Backbone=LLaMA-1-7B, Group Size=1282024.06 | 5.78 | — | — | — | |
| AWQModel Variant=Llama-1 7B, Setting=w4g128, #Bit=4.132025.03 | 5.81 | — | — | — | |
| PEQAW Bits=4, Backbone=LLAMA 7B, Group Size=None2023.05 | 5.84 | — | — | — | |
| GPTQModel Variant=Llama-1 7B, Setting=w4g128, #Bit=4.132025.03 | 5.85 | — | — | — | |
| ClusCompModel Variant=Llama-1 7B, Setting=g4n65500, #Bit=<4.142025.03 | 5.88 | — | — | 4.14 | |
| UniCuCoBackbone=Mistral-7B, Sparsity=12.5%, Latency=<1s2025.05 | 5.93 | — | — | — | |
| LUT-GEMMBits=4, Backbone=LLaMA-1-7B, Group Size=1282024.06 | 5.94 | — | — | — | |
| RTNModel Variant=Llama-1 7B, Setting=w4g128, #Bit=4.132025.03 | 5.96 | — | — | — | |
| ClusComp (optimized)Model Variant=Llama-1 7B, Setting=g6n65500, #Bit=<2.892025.03 | 6.01 | — | — | 2.89 | |
| CALDERABackbone=LLaMa-2 13B, Rank=64, B_L (= B_R)=4, Avg Bits=2.08, Quantization Bits (B_Q)=2, Evaluation Protocol=Zero-shot2024.05 | 6.04 | — | — | — | |
| Baichuan-2 7BPruning Type=Original, Sparsity=0%2026.02 | 6.04 | — | — | — | |
| QuIP# (No FT)Backbone=LLaMa-2 13B, Rank=0, B_L (= B_R)=0, Avg Bits=2, Quantization Bits (B_Q)=2, Evaluation Protocol=Zero-shot2024.05 | 6.06 | — | — | — | |
| SliM-LLM+Model Variant=Llama-1 7B, Setting=w3g128, #Bit=3.132025.03 | 6.07 | — | — | — | |
| DuQuantRatio=∼ 8×, Avg bits=4.00, Weight Updation=Yes2025.11 | 6.08 | — | — | — | |
| QATW Bits=3, Backbone=LLAMA 7B, Group Size=None2023.05 | 6.14 | — | — | — | |
| CALDERABackbone=LLaMa-2 7B, Rank=256, B_L (= B_R)=4, Avg Bits=2.4, Quantization Bits (B_Q)=2, Evaluation Protocol=Zero-shot2024.05 | 6.19 | — | — | — | |
| PEQAW Bits=3, Backbone=LLAMA 7B, Group Size=None2023.05 | 6.19 | — | — | — | |
| OPTQBits=4, Backbone=LLaMA-1-7B, Group Size=1282024.06 | 6.22 | — | — | — | |
| DuQuant*Ratio=∼ 8×, Avg bits=4.01, Weight Updation=No2025.11 | 6.28 | — | — | — | |
| QuIPModel Variant=Llama-2 70B, Setting=w2, #Bit=22025.03 | 6.33 | — | — | — | |
| Wanda*Ratio=∼ 8×, Avg bits=8.00, Weight Updation=No2025.11 | 6.42 | — | — | — | |
| ShiftAddLLMBits=4, Backbone=LLaMA-3-8B, Group Size=1282024.06 | 6.46 | — | — | — | |
| BF16Bits=16, Model=Llama 3.1 8B Instruct, Context Length=81922024.06 | 6.5 | — | — | — | |
| QTIPBits=4, Model=Llama 3.1 8B Instruct, Context Length=81922024.06 | 6.61 | — | — | — | |
| SliceGPTBase Model=LLaMA-2 13B, Pruning Type=Width, Sparsity=25%2026.02 | 6.63 | — | — | — | |
| ClusComp (optimized)Model Variant=Llama-1 7B, Setting=g8n65500, #Bit=<2.292025.03 | 6.66 | — | — | 2.29 | |
| Baichuan-2 13BPruning Type=Original, Sparsity=0%2026.02 | 6.66 | — | — | — | |
| FlattenGPTBase Model=LLaMA-2 13B, Pruning Type=Depth, Sparsity=24.37%2026.02 | 6.68 | — | — | — | |
| ClusCompModel Variant=Llama-1 7B, Setting=g6n65500, #Bit=<2.892025.03 | 6.74 | — | — | 2.89 | |
| CALDERABackbone=LLaMa-2 7B, Rank=128, B_L (= B_R)=4, Avg Bits=2.2, Quantization Bits (B_Q)=2, Evaluation Protocol=Zero-shot2024.05 | 6.76 | — | — | — | |
| QTIPBits=3, Model=Llama 3.1 8B Instruct, Context Length=81922024.06 | 6.8 | — | — | — | |
| LUT-GEMMBits=4, Backbone=LLaMA-3-8B, Group Size=1282024.06 | 6.85 | — | — | — | |
| SliceGPTBase Model=LLaMA-2 7B, Pruning Type=Width, Sparsity=21.45%2026.02 | 7.02 | — | — | — | |
| ClusCompModel Variant=Llama-1 7B, Setting=g8, #Bit=<2.152025.03 | 7.06 | — | — | 2.15 | |
| LoRA + OPTQW Bits=4, Backbone=LLAMA 7B, LoRA configuration=QV4, Group Size=None2023.05 | 7.13 | — | — | — | |
| Cosine (Window)Backbone=Mistral-7B, Sparsity=12.5%, Latency=8s2025.05 | 7.19 | — | — | — | |
| ShortGPTBackbone=Mistral-7B, Sparsity=12.5%, Latency=<1s2025.05 | 7.19 | — | — | — | |
| Weight SubcloningBackbone=Mistral-7B, Sparsity=12.5%, Latency=<1s2025.05 | 7.19 | — | — | — | |
| GQA (Original)Rank=64, KV Save=93.75, Model=Llama-3.1-8B-Instruct, One-shot evaluation=true, Calibration dataset=alpaca, Sequence length=2048, Calibration samples=2562026.03 | 7.21 | — | — | — | |
| LoRA + OPTQW Bits=3, Backbone=LLaMA 13B, LoRA configuration=QV4, Group Size=None2023.05 | 7.33 | — | — | — |