Zero-shot Classification on WinoGrande, PiQA, HellaSwag, ARC-easy, ARC-challenge, and BoolQ
76.88Avg Zero-shot AccFull Precision
Evaluation Results
| Method | Links | |
|---|---|---|
| Full PrecisionBackbone=Llama3-70B, Mixed-Precision=false, Bits=16, Quantizer Type=Uniform2026.02 | 76.88 | |
| GuidedQuantBackbone=Llama3-70B, Mixed-Precision=false, Bits=3, Quantizer Type=Non-uniform2026.02 | 75.51 | |
| ScaleBITS + RTNBackbone=Llama3-70B, Mixed-Precision=true, Bits=3.1, Quantizer Type=Uniform2026.02 | 75 | |
| GPTQ-g128Backbone=Llama3-70B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.02 | 73.72 | |
| Full PrecisionBackbone=Llama3-8B, Mixed-Precision=false, Bits=16, Quantizer Type=Uniform2026.02 | 70.69 | |
| GuidedQuantBackbone=Llama3-8B, Mixed-Precision=false, Bits=3, Quantizer Type=Non-uniform2026.02 | 68.58 | |
| ScaleBITS + RTNBackbone=Llama3-8B, Mixed-Precision=true, Bits=3.1, Quantizer Type=Uniform2026.02 | 67.87 | |
| Full PrecisionBackbone=Llama2-7B, Mixed-Precision=false, Bits=16, Quantizer Type=Uniform2026.02 | 66.55 | |
| RTN-g128Backbone=Llama3-70B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.02 | 66.33 | |
| SlimLLM-g128Backbone=Llama3-8B, Mixed-Precision=true, Bits=3.1, Quantizer Type=Uniform2026.02 | 66.03 | |
| ScaleBITS + RTNBackbone=Llama2-7B, Mixed-Precision=true, Bits=3.1, Quantizer Type=Uniform2026.02 | 65.53 | |
| GuidedQuantBackbone=Llama2-7B, Mixed-Precision=false, Bits=3, Quantizer Type=Non-uniform2026.02 | 65.12 | |
| GPTQ-g128Backbone=Llama3-8B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.02 | 64.97 | |
| ScaleBITS + RTNBackbone=Llama3-70B, Mixed-Precision=true, Bits=2.1, Quantizer Type=Uniform2026.02 | 64.72 | |
| GuidedQuantBackbone=Llama3-70B, Mixed-Precision=false, Bits=2, Quantizer Type=Non-uniform2026.02 | 64.39 | |
| GPTQ-g128Backbone=Llama2-7B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.02 | 64.15 | |
| RTN-g128Backbone=Llama2-7B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.02 | 63.65 | |
| SlimLLM-g128Backbone=Llama2-7B, Mixed-Precision=true, Bits=3.1, Quantizer Type=Uniform2026.02 | 63.06 | |
| RTN-g128Backbone=Llama3-8B, Mixed-Precision=false, Bits=3.1, Quantizer Type=Uniform2026.02 | 60.39 | |
| ScaleBITS + RTNBackbone=Llama2-7B, Mixed-Precision=true, Bits=2.1, Quantizer Type=Uniform2026.02 | 59.53 | |
| ScaleBITS + RTNBackbone=Llama3-8B, Mixed-Precision=true, Bits=2.1, Quantizer Type=Uniform2026.02 | 58.53 | |
| GuidedQuantBackbone=Llama3-8B, Mixed-Precision=false, Bits=2, Quantizer Type=Non-uniform2026.02 | 56.21 | |
| GuidedQuantBackbone=Llama2-7B, Mixed-Precision=false, Bits=2, Quantizer Type=Non-uniform2026.02 | 51.35 | |
| SlimLLM-g128Backbone=Llama2-7B, Mixed-Precision=true, Bits=2.1, Quantizer Type=Uniform2026.02 | 47 | |
| GPTQ-g128Backbone=Llama3-70B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.02 | 46.6 | |
| GPTQ-g128Backbone=Llama2-7B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.02 | 40.67 | |
| RTN-g128Backbone=Llama3-8B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.02 | 37.16 | |
| SlimLLM-g128Backbone=Llama3-8B, Mixed-Precision=true, Bits=2.1, Quantizer Type=Uniform2026.02 | 37.13 | |
| GPTQ-g128Backbone=Llama3-8B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.02 | 35.82 | |
| RTN-g128Backbone=Llama2-7B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.02 | 35.8 | |
| RTN-g128Backbone=Llama3-70B, Mixed-Precision=false, Bits=2.1, Quantizer Type=Uniform2026.02 | 35.61 |