# Language Modeling on WikiText-2, PTB3, and WikiText-103 Source: [Language Modeling on WikiText-2, PTB3, and WikiText-103]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:57:58.253Z Evaluation of language modeling performance aggregated across the WikiText-2, Penn Treebank (PTB3), and WikiText-103 datasets, typically measured by perplexity. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Language Modeling | | Dataset (including any stated split) | WikiText2, PTB3, WikiText103 | | Metric | Perplexity (WikiText-2) | | Unit | SCALAR | | Direction | Lower is better | | Results loaded | 30 of 30 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Perplexity (WikiText-2)](): Lower is better; unit: SCALAR. - [Perplexity (PTB3)](): Lower is better; unit: SCALAR. - [Perplexity (WikiText-103)](): Lower is better; unit: SCALAR. - [Average Perplexity](): Lower is better; unit: SCALAR. ## Recorded results | Method | Perplexity (WikiText-2) | Result date | Report paper | | --- | --- | --- | --- | | SASQ | 4.645 | Not recorded | 6942392745614693628506b6 | | Baseline | 4.884 | Not recorded | 6942392745614693628506b6 | | QuaRot | 4.9 | Not recorded | 6942392745614693628506b6 | | SMQ Dynamic | 4.927 | Not recorded | 6942392745614693628506b6 | | AWQ | 4.97 | Not recorded | 6942392745614693628506b6 | | SASQ | 5.182 | Not recorded | 6942392745614693628506b6 | | SASQ | 5.214 | Not recorded | 6942392745614693628506b6 | | Baseline | 5.473 | Not recorded | 6942392745614693628506b6 | | QuaRot | 5.5 | Not recorded | 6942392745614693628506b6 | | SMQ Dynamic | 5.51 | Not recorded | 6942392745614693628506b6 | | AWQ | 5.6 | Not recorded | 6942392745614693628506b6 | | Baseline | 5.683 | Not recorded | 6942392745614693628506b6 | | SpinQuant | 5.7 | Not recorded | 6942392745614693628506b6 | | SMQ Dynamic | 5.716 | Not recorded | 6942392745614693628506b6 | | AWQ | 5.78 | Not recorded | 6942392745614693628506b6 | | SASQ | 6.124 | Not recorded | 6942392745614693628506b6 | | Baseline | 6.14 | Not recorded | 6942392745614693628506b6 | | SMQ Dynamic | 6.243 | Not recorded | 6942392745614693628506b6 | | SpinQuant | 6.5 | Not recorded | 6942392745614693628506b6 | | SASQ | 6.526 | Not recorded | 6942392745614693628506b6 | | Baseline | 6.85 | Not recorded | 6942392745614693628506b6 | | SMQ Dynamic | 6.994 | Not recorded | 6942392745614693628506b6 | | LLM-QAT | 7.7 | Not recorded | 6942392745614693628506b6 | | SASQ | 8.465 | Not recorded | 6942392745614693628506b6 | | Baseline | 9.27 | Not recorded | 6942392745614693628506b6 | | SMQ Dynamic | 9.425 | Not recorded | 6942392745614693628506b6 | | LLM-QAT | 10.3 | Not recorded | 6942392745614693628506b6 | | LLM-QAT | 11.4 | Not recorded | 6942392745614693628506b6 | | LLM-QAT | 14.5 | Not recorded | 6942392745614693628506b6 | | SMQ Static | 448.524 | Not recorded | 6942392745614693628506b6 | ### SASQ Perplexity (WikiText-2): 4.645. Result ID: 6996be1c5698665ec4090fa9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA2-13B"}. This record does not assert independent reproduction. ### Baseline Perplexity (WikiText-2): 4.884. Result ID: 6996be1c5698665ec4090fa4. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA2-13B","Precision":"FP16"}. This record does not assert independent reproduction. ### QuaRot Perplexity (WikiText-2): 4.9. Result ID: 6996be1c5698665ec4090fa6. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"LLaMA2-13B"}. This record does not assert independent reproduction. ### SMQ Dynamic Perplexity (WikiText-2): 4.927. Result ID: 6996be1c5698665ec4090fa5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA2-13B"}. This record does not assert independent reproduction. ### AWQ Perplexity (WikiText-2): 4.97. Result ID: 6996be1c5698665ec4090fa7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA2-13B"}. This record does not assert independent reproduction. ### SASQ Perplexity (WikiText-2): 5.182. Result ID: 6996be1c5698665ec4090f9b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA1-7B"}. This record does not assert independent reproduction. ### SASQ Perplexity (WikiText-2): 5.214. Result ID: 6996be1c5698665ec4090fa3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA2-7B"}. This record does not assert independent reproduction. ### Baseline Perplexity (WikiText-2): 5.473. Result ID: 6996be1c5698665ec4090f9c. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA2-7B","Precision":"FP16"}. This record does not assert independent reproduction. ### QuaRot Perplexity (WikiText-2): 5.5. Result ID: 6996be1c5698665ec4090f9f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"LLaMA2-7B"}. This record does not assert independent reproduction. ### SMQ Dynamic Perplexity (WikiText-2): 5.51. Result ID: 6996be1c5698665ec4090f9d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA2-7B"}. This record does not assert independent reproduction. ### AWQ Perplexity (WikiText-2): 5.6. Result ID: 6996be1c5698665ec4090fa0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA2-7B"}. This record does not assert independent reproduction. ### Baseline Perplexity (WikiText-2): 5.683. Result ID: 6996be1c5698665ec4090f97. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA1-7B","Precision":"FP16"}. This record does not assert independent reproduction. ### SpinQuant Perplexity (WikiText-2): 5.7. Result ID: 6996be1c5698665ec4090fa2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA2-7B"}. This record does not assert independent reproduction. ### SMQ Dynamic Perplexity (WikiText-2): 5.716. Result ID: 6996be1c5698665ec4090f98. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA1-7B"}. This record does not assert independent reproduction. ### AWQ Perplexity (WikiText-2): 5.78. Result ID: 6996be1c5698665ec4090f99. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA1-7B"}. This record does not assert independent reproduction. ### SASQ Perplexity (WikiText-2): 6.124. Result ID: 6996be1c5698665ec4090fae. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA3-8B"}. This record does not assert independent reproduction. ### Baseline Perplexity (WikiText-2): 6.14. Result ID: 6996be1c5698665ec4090faa. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA3-8B","Precision":"FP16"}. This record does not assert independent reproduction. ### SMQ Dynamic Perplexity (WikiText-2): 6.243. Result ID: 6996be1c5698665ec4090fab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA3-8B"}. This record does not assert independent reproduction. ### SpinQuant Perplexity (WikiText-2): 6.5. Result ID: 6996be1c5698665ec4090fad. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA3-8B"}. This record does not assert independent reproduction. ### SASQ Perplexity (WikiText-2): 6.526. Result ID: 6996be1c5698665ec4090fb4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B"}. This record does not assert independent reproduction. ### Baseline Perplexity (WikiText-2): 6.85. Result ID: 6996be1c5698665ec4090fb2. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B","Precision":"FP16"}. This record does not assert independent reproduction. ### SMQ Dynamic Perplexity (WikiText-2): 6.994. Result ID: 6996be1c5698665ec4090fb3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B"}. This record does not assert independent reproduction. ### LLM-QAT Perplexity (WikiText-2): 7.7. Result ID: 6996be1c5698665ec4090fac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA3-8B"}. This record does not assert independent reproduction. ### SASQ Perplexity (WikiText-2): 8.465. Result ID: 6996be1c5698665ec4090fb1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-1.5B"}. This record does not assert independent reproduction. ### Baseline Perplexity (WikiText-2): 9.27. Result ID: 6996be1c5698665ec4090faf. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-1.5B","Precision":"FP16"}. This record does not assert independent reproduction. ### SMQ Dynamic Perplexity (WikiText-2): 9.425. Result ID: 6996be1c5698665ec4090fb0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-1.5B"}. This record does not assert independent reproduction. ### LLM-QAT Perplexity (WikiText-2): 10.3. Result ID: 6996be1c5698665ec4090f9a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA1-7B"}. This record does not assert independent reproduction. ### LLM-QAT Perplexity (WikiText-2): 11.4. Result ID: 6996be1c5698665ec4090fa1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA2-7B"}. This record does not assert independent reproduction. ### LLM-QAT Perplexity (WikiText-2): 14.5. Result ID: 6996be1c5698665ec4090fa8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA2-13B"}. This record does not assert independent reproduction. ### SMQ Static Perplexity (WikiText-2): 448.524. Result ID: 6996be1c5698665ec4090f9e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA2-7B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.