Language Modeling on WikiText-2 2017 (test)
7.5PPL (Uniform)FP16
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| FP16Model=Qwen3-32B2026.04 | 7.5 | — | — | — | |
| UniformModel=Qwen3-32B, Average bit-width (b_bar)=4.5, Bit-width range (b_min/b_max)=3/62026.04 | 7.52 | — | — | 0.02 | |
| UniformModel=Qwen3-32B, Average bit-width (b_bar)=4.0, Bit-width range (b_min/b_max)=3/62026.04 | 7.53 | — | — | 0.03 | |
| UniformModel=Qwen3-32B, Average bit-width (b_bar)=3.5, Bit-width range (b_min/b_max)=3/52026.04 | 7.7 | — | — | 0.2 | |
| FP16Model=Qwen3-8B2026.04 | 9.53 | — | — | — | |
| UniformModel=Qwen3-8B, Average bit-width (b_bar)=4.5, Bit-width range (b_min/b_max)=3/62026.04 | 9.61 | — | — | 0.08 | |
| UniformModel=Qwen3-8B, Average bit-width (b_bar)=4.0, Bit-width range (b_min/b_max)=3/62026.04 | 9.94 | — | — | 0.41 | |
| UniformModel=Qwen3-8B, Average bit-width (b_bar)=3.5, Bit-width range (b_min/b_max)=3/52026.04 | 10 | — | — | 0.47 | |
| FP16Model=Qwen3-4B2026.04 | 13.19 | — | — | — | |
| UniformModel=Qwen3-4B, Average bit-width (b_bar)=4.5, Bit-width range (b_min/b_max)=3/62026.04 | 13.42 | — | — | 0.23 | |
| UniformModel=Qwen3-4B, Average bit-width (b_bar)=3.5, Bit-width range (b_min/b_max)=3/52026.04 | 13.89 | — | — | 0.7 | |
| UniformModel=Qwen3-4B, Average bit-width (b_bar)=4.0, Bit-width range (b_min/b_max)=3/62026.04 | 13.92 | — | — | 0.73 | |
| RATEQUANTModel=Qwen3-4B, Average bit-width (b_bar)=3.5, Bit-width range (b_min/b_max)=3/52026.04 | — | 13.7 | 0.2 | — | |
| RATEQUANTModel=Qwen3-4B, Average bit-width (b_bar)=4.0, Bit-width range (b_min/b_max)=3/62026.04 | — | 13.47 | 0.45 | — | |
| RATEQUANTModel=Qwen3-4B, Average bit-width (b_bar)=4.5, Bit-width range (b_min/b_max)=3/62026.04 | — | 13.47 | 0.05 | — | |
| RATEQUANTModel=Qwen3-8B, Average bit-width (b_bar)=3.5, Bit-width range (b_min/b_max)=3/52026.04 | — | 9.76 | 0.24 | — | |
| RATEQUANTModel=Qwen3-8B, Average bit-width (b_bar)=4.0, Bit-width range (b_min/b_max)=3/62026.04 | — | 9.67 | 0.27 | — | |
| RATEQUANTModel=Qwen3-8B, Average bit-width (b_bar)=4.5, Bit-width range (b_min/b_max)=3/62026.04 | — | 9.59 | 0.02 | — | |
| RATEQUANTModel=Qwen3-32B, Average bit-width (b_bar)=3.5, Bit-width range (b_min/b_max)=3/52026.04 | — | 7.64 | 0.06 | — | |
| RATEQUANTModel=Qwen3-32B, Average bit-width (b_bar)=4.0, Bit-width range (b_min/b_max)=3/62026.04 | — | 7.5 | 0.03 | — | |
| RATEQUANTModel=Qwen3-32B, Average bit-width (b_bar)=4.5, Bit-width range (b_min/b_max)=3/62026.04 | — | 7.5 | 0.02 | — |