List your company
ResearchFormerly WizWand

Language Modeling on WikiText-2 context length 2048 (val)

9.92WikiText-2 PPLWaterSIC
-29.6832237.6384504.96772.2816Mar 5, 2026
Updated 8mo ago

Evaluation results

MethodLinks
WaterSICAvg. Bitwidth=4.002026.039.92
WaterSICAvg. Bitwidth=3.762026.039.99
NestQuant (W-only)Avg. Bitwidth=3.992026.0310.06
WaterSICAvg. Bitwidth=3.502026.0310.11
QTIPAvg. Bitwidth=4.022026.0310.12
NestQuant (W-only)Avg. Bitwidth=3.762026.0310.18
WaterSICAvg. Bitwidth=3.182026.0310.35
NestQuant (W-only)Avg. Bitwidth=3.502026.0310.38
WaterSICAvg. Bitwidth=3.002026.0310.57
Huffman-GPTQAvg. Bitwidth=4.012026.0310.66
AWQ (g128)Avg. Bitwidth=4.252026.0310.84
NestQuant (W-only)Avg. Bitwidth=3.182026.0310.85
QTIPAvg. Bitwidth=3.022026.0311.17
Huffman-GPTQAvg. Bitwidth=3.452026.0311.65
WaterSICAvg. Bitwidth=2.502026.0311.83
Huffman-GPTQAvg. Bitwidth=2.972026.0313.99
WaterSICAvg. Bitwidth=2.002026.0316.19
AWQ (g128)Avg. Bitwidth=3.252026.0316.74
Huffman-GPTQAvg. Bitwidth=2.542026.0317.74
QTIPAvg. Bitwidth=2.022026.0318.67
WaterSICAvg. Bitwidth=1.502026.0330.73
WaterSICAvg. Bitwidth=1.002026.0382.44
Huffman-GPTQAvg. Bitwidth=1.942026.0386.8
Huffman-GPTQAvg. Bitwidth=1.522026.031,000