Language Modeling on C4 (LLaMA Model Perplexity Evaluation)
7.26Perplexity (LLaMA-2 7B/8B)Full Precision
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Full PrecisionWeight Bits=162025.12 | 7.26 | 6.73 | 9.45 | |
| OursOutput Alignment (OA)=true, Block Size=128, Weight Bits=1.062025.12 | 19.25 | 13.8 | 35.14 | |
| ARB-RCWeight Alignment (WA)=true, Block Size=128, Weight Bits=1.062025.12 | 20.4 | 14.77 | 36.04 | |
| ARB-XOutput Alignment (OA)=true, Block Size=128, Weight Bits=1.062025.12 | 28.02 | 19.82 | 41.86 | |
| BiLLMWeight Alignment (WA)=true, Block Size=128, Weight Bits=1.062025.12 | 39.38 | 25.87 | 61.04 | |
| PB-LLMWeight Alignment (WA)=true, Block Size=128, Weight Bits=1.72025.12 | 80.69 | 184.67 | 104.15 |