# Long-Context Generation on PG-19 32K context length Source: [Long-Context Generation on PG-19 32K context length]() Collection index: [llms.txt]() Data updated: 2026-06-25T07:36:32.017Z Evaluation of language models on the PG-19 dataset, specifically testing long-context generation capabilities constrained to a 32,000 token context window, measured by throughput and acceptance metrics. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Long-Context Generation | | Dataset (including any stated split) | PG-19 32K context length | | Metric | Throughput | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 32 of 32 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Throughput](): Higher is better; unit: SCALAR. - [Tau Score](): Higher is better; unit: SCALAR. - [Alpha Factor](): Higher is better; unit: SCALAR. ## Recorded results | Method | Throughput | Result date | Report paper | | --- | --- | --- | --- | | Dustin | 329.61 | Not recorded | 6a3cb5fb206542251d7ca492 | | Dustin | 278.11 | Not recorded | 6a3cb5fb206542251d7ca492 | | Dustin | 258.69 | Not recorded | 6a3cb5fb206542251d7ca492 | | Dustin | 239.76 | Not recorded | 6a3cb5fb206542251d7ca492 | | Dustin | 213.64 | Not recorded | 6a3cb5fb206542251d7ca492 | | MagicDec | 194.38 | Not recorded | 6a3cb5fb206542251d7ca492 | | Dustin | 194.36 | Not recorded | 6a3cb5fb206542251d7ca492 | | Dustin | 180.14 | Not recorded | 6a3cb5fb206542251d7ca492 | | Dustin | 178.35 | Not recorded | 6a3cb5fb206542251d7ca492 | | ClassicSD | 155.42 | Not recorded | 6a3cb5fb206542251d7ca492 | | MagicDec | 145.94 | Not recorded | 6a3cb5fb206542251d7ca492 | | Vanilla | 141.47 | Not recorded | 6a3cb5fb206542251d7ca492 | | ClassicSD | 132.62 | Not recorded | 6a3cb5fb206542251d7ca492 | | MagicDec | 127.11 | Not recorded | 6a3cb5fb206542251d7ca492 | | Vanilla | 126.35 | Not recorded | 6a3cb5fb206542251d7ca492 | | MagicDec | 110.05 | Not recorded | 6a3cb5fb206542251d7ca492 | | ClassicSD | 91.36 | Not recorded | 6a3cb5fb206542251d7ca492 | | ClassicSD | 84.8 | Not recorded | 6a3cb5fb206542251d7ca492 | | MagicDec | 80.34 | Not recorded | 6a3cb5fb206542251d7ca492 | | MagicDec | 74.7 | Not recorded | 6a3cb5fb206542251d7ca492 | | ClassicSD | 72.65 | Not recorded | 6a3cb5fb206542251d7ca492 | | ClassicSD | 68.16 | Not recorded | 6a3cb5fb206542251d7ca492 | | Vanilla | 67.42 | Not recorded | 6a3cb5fb206542251d7ca492 | | MagicDec | 63.52 | Not recorded | 6a3cb5fb206542251d7ca492 | | Vanilla | 63.13 | Not recorded | 6a3cb5fb206542251d7ca492 | | ClassicSD | 61.23 | Not recorded | 6a3cb5fb206542251d7ca492 | | MagicDec | 57.61 | Not recorded | 6a3cb5fb206542251d7ca492 | | ClassicSD | 56.04 | Not recorded | 6a3cb5fb206542251d7ca492 | | Vanilla | 44.87 | Not recorded | 6a3cb5fb206542251d7ca492 | | Vanilla | 41.96 | Not recorded | 6a3cb5fb206542251d7ca492 | | Vanilla | 33.14 | Not recorded | 6a3cb5fb206542251d7ca492 | | Vanilla | 30.49 | Not recorded | 6a3cb5fb206542251d7ca492 | ### Dustin Throughput: 329.61. Result ID: 6a3cda71fbc373592c517fa7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### Dustin Throughput: 278.11. Result ID: 6a3cda71fbc373592c517f9f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### Dustin Throughput: 258.69. Result ID: 6a3cda71fbc373592c517f97. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### Dustin Throughput: 239.76. Result ID: 6a3cda71fbc373592c517fa3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### Dustin Throughput: 213.64. Result ID: 6a3cda71fbc373592c51802b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch Size":"16","Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### MagicDec Throughput: 194.38. Result ID: 6a3cda71fbc373592c517fa6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### Dustin Throughput: 194.36. Result ID: 6a3cda71fbc373592c517f9b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### Dustin Throughput: 180.14. Result ID: 6a3cda71fbc373592c518027. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch Size":"8","Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### Dustin Throughput: 178.35. Result ID: 6a3cda71fbc373592c517f93. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### ClassicSD Throughput: 155.42. Result ID: 6a3cda71fbc373592c517fa5. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### MagicDec Throughput: 145.94. Result ID: 6a3cda71fbc373592c517fa2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### Vanilla Throughput: 141.47. Result ID: 6a3cda71fbc373592c517fa4. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### ClassicSD Throughput: 132.62. Result ID: 6a3cda71fbc373592c517fa1. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### MagicDec Throughput: 127.11. Result ID: 6a3cda71fbc373592c51802a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch Size":"16","Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### Vanilla Throughput: 126.35. Result ID: 6a3cda71fbc373592c517fa0. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### MagicDec Throughput: 110.05. Result ID: 6a3cda71fbc373592c518026. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch Size":"8","Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### ClassicSD Throughput: 91.36. Result ID: 6a3cda71fbc373592c518029. [Reporting paper]() Recorded evaluation configuration: {"Batch Size":"16","Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### ClassicSD Throughput: 84.8. Result ID: 6a3cda71fbc373592c518025. [Reporting paper]() Recorded evaluation configuration: {"Batch Size":"8","Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### MagicDec Throughput: 80.34. Result ID: 6a3cda71fbc373592c517f9e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### MagicDec Throughput: 74.7. Result ID: 6a3cda71fbc373592c517f96. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### ClassicSD Throughput: 72.65. Result ID: 6a3cda71fbc373592c517f9d. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### ClassicSD Throughput: 68.16. Result ID: 6a3cda71fbc373592c517f95. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### Vanilla Throughput: 67.42. Result ID: 6a3cda71fbc373592c518028. [Reporting paper]() Recorded evaluation configuration: {"Batch Size":"16","Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### MagicDec Throughput: 63.52. Result ID: 6a3cda71fbc373592c517f9a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### Vanilla Throughput: 63.13. Result ID: 6a3cda71fbc373592c518024. [Reporting paper]() Recorded evaluation configuration: {"Batch Size":"8","Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### ClassicSD Throughput: 61.23. Result ID: 6a3cda71fbc373592c517f99. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### MagicDec Throughput: 57.61. Result ID: 6a3cda71fbc373592c517f92. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### ClassicSD Throughput: 56.04. Result ID: 6a3cda71fbc373592c517f91. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### Vanilla Throughput: 44.87. Result ID: 6a3cda71fbc373592c517f9c. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### Vanilla Throughput: 41.96. Result ID: 6a3cda71fbc373592c517f98. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### Vanilla Throughput: 33.14. Result ID: 6a3cda71fbc373592c517f94. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### Vanilla Throughput: 30.49. Result ID: 6a3cda71fbc373592c517f90. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.