Language Modeling on WT2, PTB, and C4 Macro Average (test)
13.1PerplexityAWQ
Evaluation Results
| Method | Links | |
|---|---|---|
| AWQModel=OPT-6.7B, Quantization (bits)=5 bits, Calibration Dataset=WT2, Groupsize=32, Calibration tokens=2^172026.03 | 13.1 | |
| AWQModel=OPT-6.7B, Quantization (bits)=5 bits, Calibration Dataset=PTB, Groupsize=32, Calibration tokens=2^172026.03 | 13.1 | |
| AWQModel=OPT-6.7B, Quantization (bits)=5 bits, Calibration Dataset=C4, Groupsize=32, Calibration tokens=2^172026.03 | 13.1 | |
| TTQModel=OPT-6.7B, Quantization (bits)=5 bits, Rank (r)=0, Groupsize=322026.03 | 13.1 | |
| TTQModel=OPT-6.7B, Quantization (bits)=4 bits, Rank (r)=16, Groupsize=322026.03 | 13.1 | |
| TTQModel=OPT-6.7B, Quantization (bits)=5 bits, Rank (r)=16, Groupsize=322026.03 | 13.1 | |
| RTNModel=OPT-6.7B, Quantization (bits)=5 bits, Groupsize=322026.03 | 13.2 | |
| AWQModel=OPT-6.7B, Quantization (bits)=4 bits, Calibration Dataset=WT2, Groupsize=32, Calibration tokens=2^172026.03 | 13.2 | |
| AWQModel=OPT-6.7B, Quantization (bits)=4 bits, Calibration Dataset=PTB, Groupsize=32, Calibration tokens=2^172026.03 | 13.2 | |
| AWQModel=OPT-6.7B, Quantization (bits)=4 bits, Calibration Dataset=C4, Groupsize=32, Calibration tokens=2^172026.03 | 13.2 | |
| TTQModel=OPT-6.7B, Quantization (bits)=4 bits, Rank (r)=0, Groupsize=322026.03 | 13.2 | |
| TTQModel=OPT-6.7B, Quantization (bits)=3 bits, Rank (r)=16, Groupsize=322026.03 | 13.4 | |
| AWQModel=OPT-6.7B, Quantization (bits)=3 bits, Calibration Dataset=WT2, Groupsize=32, Calibration tokens=2^172026.03 | 13.5 | |
| AWQModel=OPT-6.7B, Quantization (bits)=3 bits, Calibration Dataset=PTB, Groupsize=32, Calibration tokens=2^172026.03 | 13.6 | |
| AWQModel=OPT-6.7B, Quantization (bits)=3 bits, Calibration Dataset=C4, Groupsize=32, Calibration tokens=2^172026.03 | 13.6 | |
| TTQModel=OPT-6.7B, Quantization (bits)=3 bits, Rank (r)=0, Groupsize=322026.03 | 13.6 | |
| RTNModel=OPT-6.7B, Quantization (bits)=4 bits, Groupsize=322026.03 | 13.7 | |
| TTQModel=OPT-2.7B, Quantization (bits)=5 bits, Rank (r)=0, Groupsize=322026.03 | 14.9 | |
| TTQModel=OPT-2.7B, Quantization (bits)=5 bits, Rank (r)=16, Groupsize=322026.03 | 14.9 | |
| RTNModel=OPT-2.7B, Quantization (bits)=5 bits, Groupsize=322026.03 | 15 | |
| AWQModel=OPT-2.7B, Quantization (bits)=5 bits, Calibration Dataset=WT2, Groupsize=32, Calibration tokens=2^172026.03 | 15 | |
| AWQModel=OPT-2.7B, Quantization (bits)=4 bits, Calibration Dataset=PTB, Groupsize=32, Calibration tokens=2^172026.03 | 15 | |
| AWQModel=OPT-2.7B, Quantization (bits)=5 bits, Calibration Dataset=PTB, Groupsize=32, Calibration tokens=2^172026.03 | 15 | |
| AWQModel=OPT-2.7B, Quantization (bits)=4 bits, Calibration Dataset=C4, Groupsize=32, Calibration tokens=2^172026.03 | 15 | |
| AWQModel=OPT-2.7B, Quantization (bits)=5 bits, Calibration Dataset=C4, Groupsize=32, Calibration tokens=2^172026.03 | 15 | |
| TTQModel=OPT-2.7B, Quantization (bits)=4 bits, Rank (r)=0, Groupsize=322026.03 | 15 | |
| TTQModel=OPT-2.7B, Quantization (bits)=4 bits, Rank (r)=16, Groupsize=322026.03 | 15 | |
| AWQModel=OPT-2.7B, Quantization (bits)=4 bits, Calibration Dataset=WT2, Groupsize=32, Calibration tokens=2^172026.03 | 15.1 | |
| TTQModel=OPT-2.7B, Quantization (bits)=3 bits, Rank (r)=16, Groupsize=322026.03 | 15.5 | |
| RTNModel=OPT-2.7B, Quantization (bits)=4 bits, Groupsize=322026.03 | 15.7 | |
| AWQModel=OPT-2.7B, Quantization (bits)=3 bits, Calibration Dataset=WT2, Groupsize=32, Calibration tokens=2^172026.03 | 15.7 | |
| AWQModel=OPT-2.7B, Quantization (bits)=3 bits, Calibration Dataset=PTB, Groupsize=32, Calibration tokens=2^172026.03 | 15.7 | |
| AWQModel=OPT-2.7B, Quantization (bits)=3 bits, Calibration Dataset=C4, Groupsize=32, Calibration tokens=2^172026.03 | 15.7 | |
| TTQModel=OPT-2.7B, Quantization (bits)=3 bits, Rank (r)=0, Groupsize=322026.03 | 15.7 | |
| TTQModel=OPT-6.7B, Quantization (bits)=2 bits, Rank (r)=16, Groupsize=322026.03 | 16.3 | |
| AWQModel=OPT-6.7B, Quantization (bits)=2 bits, Calibration Dataset=C4, Groupsize=32, Calibration tokens=2^172026.03 | 16.9 | |
| AWQModel=OPT-1.3B, Quantization (bits)=5 bits, Calibration Dataset=WT2, Groupsize=32, Calibration tokens=2^172026.03 | 17 | |
| AWQModel=OPT-1.3B, Quantization (bits)=5 bits, Calibration Dataset=PTB, Groupsize=32, Calibration tokens=2^172026.03 | 17 | |
| AWQModel=OPT-1.3B, Quantization (bits)=5 bits, Calibration Dataset=C4, Groupsize=32, Calibration tokens=2^172026.03 | 17 | |
| TTQModel=OPT-1.3B, Quantization (bits)=5 bits, Rank (r)=0, Groupsize=322026.03 | 17 | |
| TTQModel=OPT-1.3B, Quantization (bits)=5 bits, Rank (r)=16, Groupsize=322026.03 | 17 | |
| TTQModel=OPT-1.3B, Quantization (bits)=4 bits, Rank (r)=16, Groupsize=322026.03 | 17.1 | |
| AWQModel=OPT-6.7B, Quantization (bits)=2 bits, Calibration Dataset=PTB, Groupsize=32, Calibration tokens=2^172026.03 | 17.1 | |
| RTNModel=OPT-1.3B, Quantization (bits)=5 bits, Groupsize=322026.03 | 17.2 | |
| AWQModel=OPT-1.3B, Quantization (bits)=4 bits, Calibration Dataset=C4, Groupsize=32, Calibration tokens=2^172026.03 | 17.2 | |
| TTQModel=OPT-1.3B, Quantization (bits)=4 bits, Rank (r)=0, Groupsize=322026.03 | 17.2 | |
| AWQModel=OPT-6.7B, Quantization (bits)=2 bits, Calibration Dataset=WT2, Groupsize=32, Calibration tokens=2^172026.03 | 17.2 | |
| TTQModel=OPT-6.7B, Quantization (bits)=2 bits, Rank (r)=0, Groupsize=322026.03 | 17.2 | |
| AWQModel=OPT-1.3B, Quantization (bits)=4 bits, Calibration Dataset=WT2, Groupsize=32, Calibration tokens=2^172026.03 | 17.3 | |
| AWQModel=OPT-1.3B, Quantization (bits)=4 bits, Calibration Dataset=PTB, Groupsize=32, Calibration tokens=2^172026.03 | 17.3 | |
| TTQModel=OPT-1.3B, Quantization (bits)=3 bits, Rank (r)=16, Groupsize=322026.03 | 17.9 | |
| AWQModel=OPT-1.3B, Quantization (bits)=3 bits, Calibration Dataset=WT2, Groupsize=32, Calibration tokens=2^172026.03 | 18 | |
| AWQModel=OPT-1.3B, Quantization (bits)=3 bits, Calibration Dataset=C4, Groupsize=32, Calibration tokens=2^172026.03 | 18 | |
| RTNModel=OPT-1.3B, Quantization (bits)=4 bits, Groupsize=322026.03 | 18.1 | |
| AWQModel=OPT-1.3B, Quantization (bits)=3 bits, Calibration Dataset=PTB, Groupsize=32, Calibration tokens=2^172026.03 | 18.1 | |
| TTQModel=OPT-1.3B, Quantization (bits)=3 bits, Rank (r)=0, Groupsize=322026.03 | 18.2 | |
| TTQModel=OPT-2.7B, Quantization (bits)=2 bits, Rank (r)=16, Groupsize=322026.03 | 21.2 | |
| AWQModel=OPT-2.7B, Quantization (bits)=2 bits, Calibration Dataset=C4, Groupsize=32, Calibration tokens=2^172026.03 | 22.9 | |
| AWQModel=OPT-2.7B, Quantization (bits)=2 bits, Calibration Dataset=WT2, Groupsize=32, Calibration tokens=2^172026.03 | 23.1 | |
| AWQModel=OPT-2.7B, Quantization (bits)=2 bits, Calibration Dataset=PTB, Groupsize=32, Calibration tokens=2^172026.03 | 23.2 | |
| TTQModel=OPT-2.7B, Quantization (bits)=2 bits, Rank (r)=0, Groupsize=322026.03 | 23.7 | |
| TTQModel=Qwen3-1.7B, Quantization (bits)=5 bits, Rank (r)=16, Groupsize=322026.03 | 24.1 | |
| TTQModel=Qwen3-1.7B, Quantization (bits)=5 bits, Rank (r)=0, Groupsize=322026.03 | 24.3 | |
| TTQModel=Qwen3-1.7B, Quantization (bits)=4 bits, Rank (r)=16, Groupsize=322026.03 | 24.3 | |
| AWQModel=Qwen3-1.7B, Quantization (bits)=5 bits, Calibration Dataset=WT2, Groupsize=32, Calibration tokens=2^172026.03 | 24.4 | |
| AWQModel=Qwen3-1.7B, Quantization (bits)=5 bits, Calibration Dataset=C4, Groupsize=32, Calibration tokens=2^172026.03 | 24.4 | |
| TTQModel=Qwen3-1.7B, Quantization (bits)=4 bits, Rank (r)=0, Groupsize=322026.03 | 24.4 | |
| AWQModel=Qwen3-1.7B, Quantization (bits)=4 bits, Calibration Dataset=WT2, Groupsize=32, Calibration tokens=2^172026.03 | 24.5 | |
| AWQModel=Qwen3-1.7B, Quantization (bits)=5 bits, Calibration Dataset=PTB, Groupsize=32, Calibration tokens=2^172026.03 | 24.5 | |
| AWQModel=Qwen3-1.7B, Quantization (bits)=4 bits, Calibration Dataset=PTB, Groupsize=32, Calibration tokens=2^172026.03 | 24.7 | |
| AWQModel=Qwen3-1.7B, Quantization (bits)=4 bits, Calibration Dataset=C4, Groupsize=32, Calibration tokens=2^172026.03 | 24.9 | |
| RTNModel=Qwen3-1.7B, Quantization (bits)=5 bits, Groupsize=322026.03 | 26.1 | |
| RTNModel=OPT-6.7B, Quantization (bits)=3 bits, Groupsize=322026.03 | 26.2 | |
| TTQModel=Qwen3-1.7B, Quantization (bits)=3 bits, Rank (r)=16, Groupsize=322026.03 | 26.4 | |
| RTNModel=OPT-1.3B, Quantization (bits)=3 bits, Groupsize=322026.03 | 27.2 | |
| TTQModel=OPT-1.3B, Quantization (bits)=2 bits, Rank (r)=16, Groupsize=322026.03 | 27.2 | |
| TTQModel=Qwen3-1.7B, Quantization (bits)=3 bits, Rank (r)=0, Groupsize=322026.03 | 27.3 | |
| AWQModel=Qwen3-1.7B, Quantization (bits)=3 bits, Calibration Dataset=C4, Groupsize=32, Calibration tokens=2^172026.03 | 28.2 | |
| AWQModel=Qwen3-1.7B, Quantization (bits)=3 bits, Calibration Dataset=PTB, Groupsize=32, Calibration tokens=2^172026.03 | 29.8 | |
| AWQModel=Qwen3-1.7B, Quantization (bits)=3 bits, Calibration Dataset=WT2, Groupsize=32, Calibration tokens=2^172026.03 | 30 | |
| RTNModel=Qwen3-1.7B, Quantization (bits)=4 bits, Groupsize=322026.03 | 30.6 | |
| TTQModel=OPT-125M, Quantization (bits)=5 bits, Rank (r)=16, Groupsize=322026.03 | 31.1 | |
| TTQModel=OPT-125M, Quantization (bits)=5 bits, Rank (r)=0, Groupsize=322026.03 | 31.2 | |
| AWQModel=OPT-125M, Quantization (bits)=5 bits, Calibration Dataset=PTB, Groupsize=32, Calibration tokens=2^172026.03 | 31.3 | |
| AWQModel=OPT-125M, Quantization (bits)=5 bits, Calibration Dataset=C4, Groupsize=32, Calibration tokens=2^172026.03 | 31.3 | |
| AWQModel=OPT-125M, Quantization (bits)=5 bits, Calibration Dataset=WT2, Groupsize=32, Calibration tokens=2^172026.03 | 31.4 | |
| AWQModel=OPT-1.3B, Quantization (bits)=2 bits, Calibration Dataset=C4, Groupsize=32, Calibration tokens=2^172026.03 | 31.7 | |
| RTNModel=OPT-125M, Quantization (bits)=5 bits, Groupsize=322026.03 | 31.8 | |
| TTQModel=OPT-125M, Quantization (bits)=4 bits, Rank (r)=16, Groupsize=322026.03 | 31.8 | |
| TTQModel=OPT-125M, Quantization (bits)=4 bits, Rank (r)=0, Groupsize=322026.03 | 31.9 | |
| TTQModel=Qwen3-0.6B, Quantization (bits)=5 bits, Rank (r)=0, Groupsize=322026.03 | 31.9 | |
| TTQModel=Qwen3-0.6B, Quantization (bits)=5 bits, Rank (r)=16, Groupsize=322026.03 | 31.9 | |
| AWQModel=Qwen3-0.6B, Quantization (bits)=5 bits, Calibration Dataset=PTB, Groupsize=32, Calibration tokens=2^172026.03 | 32 | |
| AWQModel=Qwen3-0.6B, Quantization (bits)=5 bits, Calibration Dataset=C4, Groupsize=32, Calibration tokens=2^172026.03 | 32.1 | |
| AWQModel=OPT-125M, Quantization (bits)=4 bits, Calibration Dataset=PTB, Groupsize=32, Calibration tokens=2^172026.03 | 32.2 | |
| TTQModel=OPT-1.3B, Quantization (bits)=2 bits, Rank (r)=0, Groupsize=322026.03 | 32.2 | |
| AWQModel=OPT-125M, Quantization (bits)=4 bits, Calibration Dataset=WT2, Groupsize=32, Calibration tokens=2^172026.03 | 32.3 | |
| AWQModel=OPT-1.3B, Quantization (bits)=2 bits, Calibration Dataset=WT2, Groupsize=32, Calibration tokens=2^172026.03 | 32.4 | |
| AWQModel=Qwen3-0.6B, Quantization (bits)=5 bits, Calibration Dataset=WT2, Groupsize=32, Calibration tokens=2^172026.03 | 32.4 | |
| AWQModel=OPT-125M, Quantization (bits)=4 bits, Calibration Dataset=C4, Groupsize=32, Calibration tokens=2^172026.03 | 32.6 |