Language Modeling on WikiText-2 (Metric: Wiki2)
4.41WikiText-2 ScoreAWQ
Evaluation results
| Method | Links | |
|---|---|---|
| AWQModel=Mixtral-8x7B, Avg bitwidth=3.252025.11 | 4.41 | |
| PuzzleMoEModel=Mixtral-8x7B, Avg bitwidth=3.352025.11 | 4.5 | |
| FP16Model backbone=Llama-2 13B2025.06 | 4.88 | |
| Qwen 2.5Memory=26.42 GiB, Size=14B, Bits=162025.05 | 4.93 | |
| NeUQIMemory=26.36 GiB, Size=72B, Bits=32025.05 | 4.99 | |
| NAEE+AWQModel=Mixtral-8x7B, Avg bitwidth=3.192025.11 | 5.1 | |
| FlatQuantWeight Quantizer=GPTQ, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 5.11 | |
| FlatQuantWeight Quantizer=RTN, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 5.12 | |
| SpinQuantWeight Quantizer=GPTQ, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 5.24 | |
| OSTQuantWeight Quantizer=GPTQ, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 5.29 | |
| OSTQuantWeight Quantizer=RTN, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 5.34 | |
| FPTQuantWeight Quantizer=GPTQ, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 5.35 | |
| FPTQuantWeight Quantizer=RTN, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 5.37 | |
| HC-SMoE+AWQModel=Mixtral-8x7B, Avg bitwidth=3.302025.11 | 5.39 | |
| QuaRotWeight Quantizer=GPTQ, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 5.4 | |
| SpinQuantWeight Quantizer=RTN, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 5.44 | |
| FP16Model backbone=Llama-2 7B2025.06 | 5.47 | |
| FlatQuantWeight Quantizer=GPTQ, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 5.78 | |
| FlatQuantWeight Quantizer=RTN, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 5.79 | |
| NeUQIMemory=11.72 GiB, Size=32B, Bits=32025.05 | 5.85 | |
| OSTQuantWeight Quantizer=GPTQ, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 5.92 | |
| SpinQuantWeight Quantizer=GPTQ, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 5.96 | |
| FPTQuantWeight Quantizer=RTN, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 5.97 | |
| FPTQuantWeight Quantizer=GPTQ, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 6.07 | |
| QuaRotWeight Quantizer=RTN, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 6.1 | |
| QuaRotWeight Quantizer=GPTQ, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 6.1 | |
| OriginalRatio=100%2025.10 | 6.14 | |
| FP16Model backbone=Llama-3 8B2025.06 | 6.14 | |
| SpinQuantWeight Quantizer=RTN, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 6.14 | |
| NWCRatio=25%2025.10 | 6.32 | |
| OSTQuantWeight Quantizer=RTN, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 6.38 | |
| Qwen 2.5Memory=13.05 GiB, Size=7B, Bits=162025.05 | 6.39 | |
| NeUQIMemory=4.96 GiB, Size=14B, Bits=32025.05 | 6.82 | |
| AWQModel=Deepseek-MoE, Avg bitwidth=3.252025.11 | 6.87 | |
| GPTQMemory=26.36 GiB, Size=72B, Bits=32025.05 | 6.89 | |
| FlatQuantWeight Quantizer=GPTQ, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 6.9 | |
| FlatQuantWeight Quantizer=RTN, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 6.98 | |
| PuzzleMoEModel=Deepseek-MoE, Avg bitwidth=3.352025.11 | 7.05 | |
| GPTQMemory=11.72 GiB, Size=32B, Bits=32025.05 | 7.21 | |
| OSTQuantWeight Quantizer=GPTQ, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 7.32 | |
| SpinQuantWeight Quantizer=GPTQ, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 7.39 | |
| Qwen 2.5Memory=5.55 GiB, Size=3B, Bits=162025.05 | 7.44 | |
| FPTQuantWeight Quantizer=GPTQ, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 7.6 | |
| FPTQuantWeight Quantizer=RTN, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 7.67 | |
| FP16 BaselineBackbone Model=LLaMA3-3B, Precision=FP16, Evaluation Protocol=Zero-shot, Training Context=QAT on FineWebEdu2026.05 | 7.7 | |
| SpinQuantWeight Quantizer=RTN, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 7.96 | |
| OSTQuantWeight Quantizer=RTN, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 7.98 | |
| SVD-LLM V2Ratio=80%2025.10 | 8.01 | |
| QuaRotWeight Quantizer=GPTQ, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 8.16 | |
| GPTQMemory=4.96 GiB, Size=14B, Bits=32025.05 | 8.48 | |
| QuaRotWeight Quantizer=RTN, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 8.56 | |
| Qwen 2.5Memory=2.62 GiB, Size=1.5B, Bits=162025.05 | 8.58 | |
| NeUQIMemory=2.45 GiB, Size=7B, Bits=32025.05 | 8.64 | |
| FP16Model=Llama-3.2 3B-it, # Bits (W-A-KV)=16-16-162025.06 | 10.48 | |
| QuaRotWeight Quantizer=RTN, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 10.6 | |
| GPTQMemory=2.45 GiB, Size=7B, Bits=32025.05 | 10.73 | |
| FlatQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.06 | 10.9 | |
| QuaRotModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.06 | 11.03 | |
| FPTQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.06 | 11.06 | |
| FlatQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.06 | 11.49 | |
| SpinQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.06 | 11.5 | |
| RTN-optModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.06 | 11.68 | |
| SVD-LLMRatio=80%2025.10 | 11.82 | |
| FPTQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.06 | 11.82 | |
| ParetoQ + JacQuantBackbone Model=LLaMA3-3B, Precision=W1.58A8, Evaluation Protocol=Zero-shot, Training Context=QAT on FineWebEdu2026.05 | 12.7 | |
| SpinQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.06 | 13.04 | |
| ParetoQBackbone Model=LLaMA3-3B, Precision=W1.58A8, Evaluation Protocol=Zero-shot, Training Context=QAT on FineWebEdu2026.05 | 13.1 | |
| QuaRotModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.06 | 13.25 | |
| ParetoQ + JacQuantBackbone Model=LLaMA3-3B, Precision=W1A8, Evaluation Protocol=Zero-shot, Training Context=QAT on FineWebEdu2026.05 | 15.2 | |
| ParetoQBackbone Model=LLaMA3-3B, Precision=W1A8, Evaluation Protocol=Zero-shot, Training Context=QAT on FineWebEdu2026.05 | 15.7 | |
| FP16 BaselineBackbone=Qwen-1.7B, Quantization=FP16, Evaluation Protocol=Zero-shot2026.05 | 16.2 | |
| JacQuantBackbone=Qwen-1.7B, Quantization=W2A8, Evaluation Protocol=Zero-shot, Training Dataset=FineWebEdu, Training Method=QAT, Baseline=ParetoQ2026.05 | 21.5 | |
| ParetoQBackbone=Qwen-1.7B, Quantization=W2A8, Evaluation Protocol=Zero-shot, Training Dataset=FineWebEdu, Training Method=QAT2026.05 | 22.2 | |
| HC-SMoE+AWQModel=Deepseek-MoE, Avg bitwidth=3.302025.11 | 26.7 | |
| SmoothQuantWeight Quantizer=RTN, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 35.9 | |
| JacQuantBackbone=Qwen-1.7B, Quantization=W1A8, Evaluation Protocol=Zero-shot, Training Dataset=FineWebEdu, Training Method=QAT, Baseline=ParetoQ2026.05 | 45.3 | |
| ParetoQBackbone=Qwen-1.7B, Quantization=W1A8, Evaluation Protocol=Zero-shot, Training Dataset=FineWebEdu, Training Method=QAT2026.05 | 46.5 | |
| RTN-optModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.06 | 64.86 | |
| SmoothQuantWeight Quantizer=RTN, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 83.1 | |
| SmoothQuantWeight Quantizer=RTN, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 210 | |
| SpinQuantBackbone Model=LLaMA3-3B, Precision=W1.58A8, Evaluation Protocol=Zero-shot, Training Context=QAT on FineWebEdu2026.05 | 3,100 | |
| GPTQBackbone Model=LLaMA3-3B, Precision=W1.58A8, Evaluation Protocol=Zero-shot, Training Context=QAT on FineWebEdu2026.05 | 270,000 | |
| RTNBackbone Model=LLaMA3-3B, Precision=W1.58A8, Evaluation Protocol=Zero-shot, Training Context=QAT on FineWebEdu2026.05 | 790,000 | |
| SpinQuantBackbone Model=LLaMA3-3B, Precision=W1A8, Evaluation Protocol=Zero-shot, Training Context=QAT on FineWebEdu2026.05 | 45,000,000 | |
| GPTQBackbone Model=LLaMA3-3B, Precision=W1A8, Evaluation Protocol=Zero-shot, Training Context=QAT on FineWebEdu2026.05 | 59,000,000 | |
| RTNBackbone Model=LLaMA3-3B, Precision=W1A8, Evaluation Protocol=Zero-shot, Training Context=QAT on FineWebEdu2026.05 | 73,000,000 |