Language Modeling on Wiki2
4.01PPLBF16
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| BF16Models=Mixtral-8x7B, Quantization Strategy=Embedding-wise, Quantization Bit-width=BF162026.04 | 4.01 | — | — | — | |
| CodeQuantModels=Mixtral-8x7B, Quantization Strategy=Embedding-wise, Quantization Bit-width=A4W42026.04 | 4.65 | — | — | — | |
| DenseModel=LLaMA2-13B, Comp. Rate=-, Method=Dense2025.12 | 4.88 | — | — | — | |
| FP16Model=LLaMA2-13B, Wbits=162025.05 | 4.88 | — | — | — | |
| DenseRatio=1.0, zero-shot evaluation=true2026.04 | 4.88 | — | — | — | |
| BF16Backbone=Llama-2-13B, Quantization=None2026.05 | 4.88 | — | — | — | |
| MBOKModel=LLaMA2-13B, Wbits=3×12025.05 | 5.35 | — | — | — | |
| FP16Model=LLaMA2-7B, Wbits=162025.05 | 5.47 | — | — | — | |
| BF16Backbone=LLaMA-2-7B, Quantization=None2026.05 | 5.47 | — | — | — | |
| DenseCompression Ratio=1.0, Backbone=LLaMA-7B2026.04 | 5.68 | — | — | — | |
| SkipCatModel=LLaMA2-13B, Comp. Rate=20%, Method=SkipCat2025.12 | 5.79 | — | — | — | |
| AA-SVDRatio=0.8, zero-shot evaluation=true2026.04 | 5.95 | — | — | — | |
| MBOKModel=LLaMA2-13B, Wbits=2×12025.05 | 5.97 | — | — | — | |
| AA-SVDCompression Ratio=0.8, Backbone=LLaMA-7B, Weight Remapping=Dobi-SVD-style2026.04 | 6.01 | — | — | — | |
| Dobi-SVDCompression Ratio=0.8, Backbone=LLaMA-7B, Rank Allocation=Dynamic/non-uniform, Weight Remapping=Dobi-SVD-style2026.04 | 6.08 | — | — | — | |
| MBOKModel=LLaMA2-7B, Wbits=3×12025.05 | 6.12 | — | — | — | |
| BF16 (dense)Model=LLaMA-3 8B2026.05 | 6.14 | — | — | — | |
| DenseRatio=1.0, Base Model=LLaMA-3-8B, Evaluation Protocol=Zero-shot2026.04 | 6.24 | — | — | — | |
| DenseModel=14B-Base2026.05 | 6.38 | — | — | — | |
| QTIPBackbone=LLaMA-7B, Sequence Length=2048, Weight Bits=2-bit2025.05 | 6.52 | — | — | — | |
| SVD-LLMRatio=0.8, zero-shot evaluation=true2026.04 | 6.65 | — | — | — | |
| BF16Models=DeepSeek-V2-Lite, Quantization Strategy=Embedding-wise, Quantization Bit-width=BF162026.04 | 6.69 | — | — | — | |
| BF16Model=DeepSeek-V2-Lite, Quantization Precision=BF16, Quantization Granularity=N/A2026.04 | 6.69 | — | — | — | |
| ADMM-QModel=14B-Base, Weight Bits=W42026.05 | 6.79 | — | — | — | |
| SVD-LLMModel=LLaMA2-13B, Comp. Rate=20%, Method=SVD-LLM2025.12 | 6.83 | — | — | — | |
| MBOKBackbone=LLaMA-7B, Sequence Length=2048, Number of Kernels=22025.05 | 6.83 | — | — | — | |
| BF16Models=Phi-mini MoE-Instruct, Quantization Strategy=Embedding-wise, Quantization Bit-width=BF162026.04 | 6.83 | — | — | — | |
| BF16Model=Phi-mini MoE-Instruct, Quantization Precision=BF16, Quantization Granularity=N/A2026.04 | 6.83 | — | — | — | |
| CodeQuantModel=DeepSeek-V2-Lite, Quantization Precision=A8W4, Quantization Granularity=Block-wise2026.04 | 6.83 | — | — | — | |
| DenseRatio=1.0, zero-shot evaluation=true2026.04 | 6.84 | — | — | — | |
| CodeQuantModel=DeepSeek-V2-Lite, Quantization Precision=A8W4, Quantization Granularity=Embedding-wise2026.04 | 6.84 | — | — | — | |
| QUIP#Backbone=LLaMA-7B, Sequence Length=2048, Weight Bits=2-bit2025.05 | 6.86 | — | — | — | |
| SqueezeLLMModel=DeepSeek-V2-Lite, Quantization Precision=A8W4, Quantization Granularity=Block-wise2026.04 | 6.86 | — | — | — | |
| MBOKModel=LLaMA2-7B, Wbits=2×12025.05 | 6.87 | — | — | — | |
| GPTQModel=14B-Base, Weight Bits=W42026.05 | 6.87 | — | — | — | |
| AA-SVDCompression Ratio=0.8, Backbone=LLaMA-7B2026.04 | 6.89 | — | — | — | |
| SqueezeLLMModel=DeepSeek-V2-Lite, Quantization Precision=A8W4, Quantization Granularity=Embedding-wise2026.04 | 6.93 | — | — | — | |
| SkipCatModel=LLaMA2-13B, Comp. Rate=30%, Method=SkipCat2025.12 | 6.99 | — | — | — | |
| Llama-3.1-8BCompression Ratio=Original2025.10 | 6.99 | — | — | — | |
| DenseModel=8B-Base2026.05 | 7 | — | — | — | |
| CodeQuantModels=DeepSeek-V2-Lite, Quantization Strategy=Block-wise, Quantization Bit-width=A4W42026.04 | 7.03 | — | — | — | |
| MoSModel=LLaMA2-13B, Wbits=12025.05 | 7.08 | — | — | — | |
| CodeQuantModels=DeepSeek-V2-Lite, Quantization Strategy=Embedding-wise, Quantization Bit-width=A4W42026.04 | 7.08 | — | — | — | |
| AA-SVDCompression Ratio=0.6, Backbone=LLaMA-7B, Weight Remapping=Dobi-SVD-style2026.04 | 7.09 | — | — | — | |
| CodeQuantModel=Phi-mini MoE-Instruct, Quantization Precision=A8W4, Quantization Granularity=Block-wise2026.04 | 7.11 | — | — | — | |
| SAES-SVDCompression Ratio=0.8, Backbone=LLaMA-7B2026.04 | 7.17 | — | — | — | |
| ADMM-QModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 7.17 | — | — | — | |
| SqueezeLLMModel=Phi-mini MoE-Instruct, Quantization Precision=A8W4, Quantization Granularity=Block-wise2026.04 | 7.18 | — | — | — | |
| ADMM-QModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 7.21 | — | — | — | |
| QuaRotModel=DeepSeek-V2-Lite, Quantization Precision=A8W4, Quantization Granularity=Block-wise2026.04 | 7.22 | — | — | — | |
| ADMM-QModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 7.24 | — | — | — | |
| ADMM-QModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 7.26 | — | — | — | |
| GPTQModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 7.26 | — | — | — | |
| GPTQModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 7.27 | — | — | — | |
| CodeQuantModels=Phi-mini MoE-Instruct, Quantization Strategy=Block-wise, Quantization Bit-width=A4W42026.04 | 7.28 | — | — | — | |
| AWQModel=14B-Base, Weight Bits=W42026.05 | 7.28 | — | — | — | |
| QuaRotModel=DeepSeek-V2-Lite, Quantization Precision=A8W4, Quantization Granularity=Embedding-wise2026.04 | 7.29 | — | — | — | |
| GPTQModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 7.34 | — | — | — | |
| GPTQModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 7.35 | — | — | — | |
| CodeQuantModel=Phi-mini MoE-Instruct, Quantization Precision=A8W4, Quantization Granularity=Embedding-wise2026.04 | 7.36 | — | — | — | |
| SqueezeLLMModel=Phi-mini MoE-Instruct, Quantization Precision=A8W4, Quantization Granularity=Embedding-wise2026.04 | 7.41 | — | — | — | |
| SmoothQuantModel=DeepSeek-V2-Lite, Quantization Precision=A8W4, Quantization Granularity=Block-wise2026.04 | 7.42 | — | — | — | |
| AA-SVDRatio=0.6, zero-shot evaluation=true2026.04 | 7.44 | — | — | — | |
| RTNModel=DeepSeek-V2-Lite, Quantization Precision=A8W4, Quantization Granularity=Block-wise2026.04 | 7.47 | — | — | — | |
| QuaRotModel=Phi-mini MoE-Instruct, Quantization Precision=A8W4, Quantization Granularity=Block-wise2026.04 | 7.48 | — | — | — | |
| OneBitModel=LLaMA2-13B, Wbits=12025.05 | 7.56 | — | — | — | |
| ADMM-QModel=8B-Base, Weight Bits=W42026.05 | 7.56 | — | — | — | |
| SmoothQuantModel=DeepSeek-V2-Lite, Quantization Precision=A8W4, Quantization Granularity=Embedding-wise2026.04 | 7.61 | — | — | — | |
| DenseModel=32B2026.05 | 7.61 | — | — | — | |
| QuaRotModels=DeepSeek-V2-Lite, Quantization Strategy=Block-wise, Quantization Bit-width=A4W42026.04 | 7.62 | — | — | — | |
| CodeQuantModels=Phi-mini MoE-Instruct, Quantization Strategy=Embedding-wise, Quantization Bit-width=A4W42026.04 | 7.63 | — | — | — | |
| QuaRotModels=Phi-mini MoE-Instruct, Quantization Strategy=Block-wise, Quantization Bit-width=A4W42026.04 | 7.63 | — | — | — | |
| QuaRotModel=Phi-mini MoE-Instruct, Quantization Precision=A8W4, Quantization Granularity=Embedding-wise2026.04 | 7.69 | — | — | — | |
| RTNModel=DeepSeek-V2-Lite, Quantization Precision=A8W4, Quantization Granularity=Embedding-wise2026.04 | 7.72 | — | — | — | |
| QuaRotModels=DeepSeek-V2-Lite, Quantization Strategy=Embedding-wise, Quantization Bit-width=A4W42026.04 | 7.75 | — | — | — | |
| CodeQuanthadModels=DeepSeek V2-Lite, Bit Width=A8W4, Quantization Granularity=Embedding-wise, Online Hadamard Transform=Enabled2026.04 | 7.8 | — | — | — | |
| LLaMA3Size=3.0B, Bit=16.02025.08 | 7.8 | — | — | — | |
| Llama-3.2-3BCompression Ratio=Original2025.10 | 7.82 | — | — | — | |
| GPTQModel=8B-Base, Weight Bits=W42026.05 | 7.82 | — | — | — | |
| MoSModel=LLaMA2-7B, Wbits=12025.05 | 7.88 | — | — | — | |
| DenseModel=4B-Base2026.05 | 7.9 | — | — | — | |
| ADMM-QModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 7.91 | — | — | — | |
| QuaRotModels=Phi-mini MoE-Instruct, Quantization Strategy=Embedding-wise, Quantization Bit-width=A4W42026.04 | 7.93 | — | — | — | |
| SVD-LLMCompression Ratio=0.8, Backbone=LLaMA-7B, LoRA fine-tuning=true2026.04 | 7.94 | — | — | — | |
| Dip-SVDCompression Ratio=0.8, Backbone=LLaMA-7B, Rank Allocation=Dynamic/non-uniform2026.04 | 7.95 | — | — | — | |
| RTNModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 7.95 | — | — | — | |
| ADMM-QModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=groupsize 1282026.05 | 8.01 | — | — | — | |
| Qwen-3-8BCompression Ratio=Original2025.10 | 8.02 | — | — | — | |
| ADMM-QModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Symmetric, Granularity=per-channel2026.05 | 8.02 | — | — | — | |
| ADMM-QModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 8.05 | — | — | — | |
| AWQModel=8B-Base, Weight Bits=W42026.05 | 8.06 | — | — | — | |
| ADMM-QModel=32B, Weight Bits=W42026.05 | 8.06 | — | — | — | |
| RTNModel=LLaMA-3 8B, Bit-width=W4A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 8.11 | — | — | — | |
| Dobi-SVDCompression Ratio=0.6, Backbone=LLaMA-7B, Rank Allocation=Dynamic/non-uniform, Weight Remapping=Dobi-SVD-style2026.04 | 8.12 | — | — | — | |
| GPTQModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=groupsize 1282026.05 | 8.14 | — | — | — | |
| ADMM-QModel=14B-Base, Weight Bits=W32026.05 | 8.16 | — | — | — | |
| GPTQModel=32B, Weight Bits=W42026.05 | 8.22 | — | — | — | |
| GPTQModel=LLaMA-3 8B, Bit-width=W3A4KV4, Scheme=Asymmetric, Granularity=per-channel2026.05 | 8.25 | — | — | — | |
| AA-SVDCompression Ratio=0.6, Backbone=LLaMA-7B2026.04 | 8.35 | — | — | — | |
| SmoothQuantModel=Phi-mini MoE-Instruct, Quantization Precision=A8W4, Quantization Granularity=Block-wise2026.04 | 8.4 | — | — | — |