Multi-task Language Understanding on MMLU (5-shot)
78.74Accuracy (5-shot)Full Precision
Evaluation Results
| Method | Links | |
|---|---|---|
| Full PrecisionBackbone=Llama3-70B, Mixed-Precision=false, Bits=16, Quantizer Type=Uniform2026.02 | 78.74 | |
| ScaleBITS + RTNBackbone=Llama3-70B, Mixed-Precision=true, Bits=3.1, Quantizer Type=Uniform2026.02 | 76.88 | |
| GuidedQuantBackbone=Llama3-70B, Mixed-Precision=false, Bits=3, Quantizer Type=Non-uniform2026.02 | 76.86 | |
| GPTQ-g128Backbone=Llama3-70B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.02 | 74.51 | |
| Full PrecisionBackbone=Llama3-8B, Mixed-Precision=false, Bits=16, Quantizer Type=Uniform2026.02 | 65.34 | |
| ScaleBITS + RTNBackbone=Llama3-8B, Mixed-Precision=true, Bits=3.1, Quantizer Type=Uniform2026.02 | 59.35 | |
| GuidedQuantBackbone=Llama3-8B, Mixed-Precision=false, Bits=3, Quantizer Type=Non-uniform2026.02 | 59.07 | |
| ScaleBITS + RTNBackbone=Llama3-70B, Mixed-Precision=true, Bits=2.1, Quantizer Type=Uniform2026.02 | 58.82 | |
| SlimLLM-g128Backbone=Llama3-8B, Mixed-Precision=true, Bits=3.1, Quantizer Type=Uniform2026.02 | 55.03 | |
| RTN-g128Backbone=Llama3-70B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.02 | 54.56 | |
| GPTQ-g128Backbone=Llama3-8B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.02 | 54.27 | |
| RTN-g128Backbone=Llama3-8B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.02 | 46.45 | |
| Full PrecisionBackbone=Llama2-7B, Mixed-Precision=false, Bits=16, Quantizer Type=Uniform2026.02 | 45.86 | |
| GuidedQuantBackbone=Llama3-70B, Mixed-Precision=false, Bits=2, Quantizer Type=Non-uniform2026.02 | 45.55 | |
| ScaleBITS + RTNBackbone=Llama2-7B, Mixed-Precision=true, Bits=3.1, Quantizer Type=Uniform2026.02 | 43.48 | |
| GuidedQuantBackbone=Llama2-7B, Mixed-Precision=false, Bits=3, Quantizer Type=Non-uniform2026.02 | 43.38 | |
| GPTQ-g128Backbone=Llama2-7B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.02 | 41.43 | |
| SlimLLM-g128Backbone=Llama2-7B, Mixed-Precision=true, Bits=3.1, Quantizer Type=Uniform2026.02 | 39.79 | |
| ScaleBITS + RTNBackbone=Llama3-8B, Mixed-Precision=true, Bits=2.1, Quantizer Type=Uniform2026.02 | 39.36 | |
| RTN-g128Backbone=Llama2-7B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.02 | 37.66 | |
| ScaleBITS + RTNBackbone=Llama2-7B, Mixed-Precision=true, Bits=2.1, Quantizer Type=Uniform2026.02 | 34.01 | |
| GuidedQuantBackbone=Llama2-7B, Mixed-Precision=false, Bits=2, Quantizer Type=Non-uniform2026.02 | 29.37 | |
| GuidedQuantBackbone=Llama3-8B, Mixed-Precision=false, Bits=2, Quantizer Type=Non-uniform2026.02 | 26.97 | |
| GPTQ-g128Backbone=Llama3-70B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.02 | 26.63 | |
| SlimLLM-g128Backbone=Llama3-8B, Mixed-Precision=true, Bits=2.1, Quantizer Type=Uniform2026.02 | 25.51 | |
| GPTQ-g128Backbone=Llama3-8B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.02 | 25.47 | |
| GPTQ-g128Backbone=Llama2-7B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.02 | 24.95 | |
| RTN-g128Backbone=Llama2-7B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.02 | 24.8 | |
| RTN-g128Backbone=Llama3-8B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.02 | 24.1 | |
| SlimLLM-g128Backbone=Llama2-7B, Mixed-Precision=true, Bits=2.1, Quantizer Type=Uniform2026.02 | 23.69 | |
| RTN-g128Backbone=Llama3-70B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.02 | 23.02 |