Zero-shot Language Modeling on LLaMA-3-1B
9.6Perplexity (PPL)FP Model
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FP ModelBackbone=LLaMA-3-1B, Weight precision=Full, Activation precision=Full2026.05 | 9.6 | 58.5 | |
| WINQBackbone=LLaMA-3-1B, Weight precision=2-bit, Activation precision=4-bit, Hadamard transform=true2026.05 | 16.9 | 49.3 | |
| QuESTBackbone=LLaMA-3-1B, Weight precision=2-bit, Activation precision=4-bit, Hadamard transform=false2026.05 | 17.4 | 48.6 | |
| WINQBackbone=LLaMA-3-1B, Weight precision=1-bit, Activation precision=4-bit, Hadamard transform=true2026.05 | 42.3 | 43 | |
| QuESTBackbone=LLaMA-3-1B, Weight precision=1-bit, Activation precision=4-bit, Hadamard transform=false2026.05 | 42.9 | 42.4 |