Language Modeling on Wikitext 2 (PPL and Hardware Metrics)
4.67Perplexity (Wikitext-2)SAGE-PTQ
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SAGE-PTQModel=LLaMA-2 70B, Weight Bits=1.03, Scale Bits=0.004, Block Size=N/A2026.06 | 4.67 | 6.46 | |
| SAGE-PTQModel=LLaMA-1 65B, Weight Bits=1.03, Scale Bits=0.004, Block Size=N/A2026.06 | 4.93 | 6.46 | |
| BiLLMModel=LLaMA-2 70B, Weight Bits=1.08, Scale Bits=1.00, Block Size=1282026.06 | 8.41 | 13 | |
| BiLLMModel=LLaMA-1 65B, Weight Bits=1.09, Scale Bits=1.00, Block Size=1282026.06 | 8.49 | 13.06 | |
| GPTQModel=LLaMA-1 65B, Weight Bits=2.00, Scale Bits=0.25, Block Size=1282026.06 | 8.78 | 14.06 | |
| GPTQModel=LLaMA-2 70B, Weight Bits=2.00, Scale Bits=0.25, Block Size=1282026.06 | 9.12 | 14.06 | |
| SAGE-PTQModel=OPT 66B, Weight Bits=1.07, Scale Bits=0.009, Block Size=N/A2026.06 | 11.31 | 6.7 | |
| BiLLMModel=OPT 66B, Weight Bits=1.11, Scale Bits=1.00, Block Size=1282026.06 | 12.06 | 13.19 | |
| PB-LLM (10%)Model=LLaMA-1 65B, Weight Bits=1.70, Scale Bits=0.50, Block Size=1282026.06 | 12.53 | 13.75 | |
| PB-LLM (10%)Model=LLaMA-2 70B, Weight Bits=1.70, Scale Bits=0.50, Block Size=1282026.06 | 28.37 | 13.75 | |
| PB-LLM (10%)Model=OPT 66B, Weight Bits=1.70, Scale Bits=0.50, Block Size=1282026.06 | 29.09 | 13.75 | |
| GPTQModel=OPT 66B, Weight Bits=2.00, Scale Bits=0.25, Block Size=1282026.06 | 82.1 | 14.06 |