Language Modeling on WikiText-2 context length 2048 (val)
9.92WikiText-2 PPLWaterSIC
Evaluation Results
| Method | Links | |
|---|---|---|
| WaterSICAvg. Bitwidth=4.002026.03 | 9.92 | |
| WaterSICAvg. Bitwidth=3.762026.03 | 9.99 | |
| NestQuant (W-only)Avg. Bitwidth=3.992026.03 | 10.06 | |
| WaterSICAvg. Bitwidth=3.502026.03 | 10.11 | |
| QTIPAvg. Bitwidth=4.022026.03 | 10.12 | |
| NestQuant (W-only)Avg. Bitwidth=3.762026.03 | 10.18 | |
| WaterSICAvg. Bitwidth=3.182026.03 | 10.35 | |
| NestQuant (W-only)Avg. Bitwidth=3.502026.03 | 10.38 | |
| WaterSICAvg. Bitwidth=3.002026.03 | 10.57 | |
| Huffman-GPTQAvg. Bitwidth=4.012026.03 | 10.66 | |
| AWQ (g128)Avg. Bitwidth=4.252026.03 | 10.84 | |
| NestQuant (W-only)Avg. Bitwidth=3.182026.03 | 10.85 | |
| QTIPAvg. Bitwidth=3.022026.03 | 11.17 | |
| Huffman-GPTQAvg. Bitwidth=3.452026.03 | 11.65 | |
| WaterSICAvg. Bitwidth=2.502026.03 | 11.83 | |
| Huffman-GPTQAvg. Bitwidth=2.972026.03 | 13.99 | |
| WaterSICAvg. Bitwidth=2.002026.03 | 16.19 | |
| AWQ (g128)Avg. Bitwidth=3.252026.03 | 16.74 | |
| Huffman-GPTQAvg. Bitwidth=2.542026.03 | 17.74 | |
| QTIPAvg. Bitwidth=2.022026.03 | 18.67 | |
| WaterSICAvg. Bitwidth=1.502026.03 | 30.73 | |
| WaterSICAvg. Bitwidth=1.002026.03 | 82.44 | |
| Huffman-GPTQAvg. Bitwidth=1.942026.03 | 86.8 | |
| Huffman-GPTQAvg. Bitwidth=1.522026.03 | 1,000 |