# Long-Context Generation on PG-19 16K context length Source: [Long-Context Generation on PG-19 16K context length]() Collection index: [llms.txt]() Data updated: 2026-06-25T07:36:31.920Z Evaluation of language models on the PG-19 dataset, specifically testing long-context generation capabilities constrained to a 16,000 token context window. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Long-Context Generation | | Dataset (including any stated split) | PG-19 16K context length | | Metric | Throughput | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 32 of 32 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Throughput](): Higher is better; unit: SCALAR. - [Tau (τ)](): Higher is better; unit: SCALAR. - [Alpha (α) Factor](): Higher is better; unit: SCALAR. ## Recorded results | Method | Throughput | Result date | Report paper | | --- | --- | --- | --- | | Dustin | 486.5 | Not recorded | 6a3cb5fb206542251d7ca492 | | Dustin | 397.19 | Not recorded | 6a3cb5fb206542251d7ca492 | | Dustin | 379.82 | Not recorded | 6a3cb5fb206542251d7ca492 | | Dustin | 351.17 | Not recorded | 6a3cb5fb206542251d7ca492 | | MagicDec | 296.98 | Not recorded | 6a3cb5fb206542251d7ca492 | | Dustin | 280.57 | Not recorded | 6a3cb5fb206542251d7ca492 | | ClassicSD | 270.66 | Not recorded | 6a3cb5fb206542251d7ca492 | | Dustin | 264.58 | Not recorded | 6a3cb5fb206542251d7ca492 | | Vanilla | 249.36 | Not recorded | 6a3cb5fb206542251d7ca492 | | Dustin | 221.47 | Not recorded | 6a3cb5fb206542251d7ca492 | | MagicDec | 217.74 | Not recorded | 6a3cb5fb206542251d7ca492 | | Vanilla | 206.22 | Not recorded | 6a3cb5fb206542251d7ca492 | | MagicDec | 202.91 | Not recorded | 6a3cb5fb206542251d7ca492 | | Dustin | 200.55 | Not recorded | 6a3cb5fb206542251d7ca492 | | ClassicSD | 194.8 | Not recorded | 6a3cb5fb206542251d7ca492 | | MagicDec | 171.8 | Not recorded | 6a3cb5fb206542251d7ca492 | | ClassicSD | 168.33 | Not recorded | 6a3cb5fb206542251d7ca492 | | ClassicSD | 149.5 | Not recorded | 6a3cb5fb206542251d7ca492 | | MagicDec | 137.91 | Not recorded | 6a3cb5fb206542251d7ca492 | | ClassicSD | 130.65 | Not recorded | 6a3cb5fb206542251d7ca492 | | MagicDec | 127.52 | Not recorded | 6a3cb5fb206542251d7ca492 | | Vanilla | 124.43 | Not recorded | 6a3cb5fb206542251d7ca492 | | ClassicSD | 122.97 | Not recorded | 6a3cb5fb206542251d7ca492 | | Vanilla | 110.4 | Not recorded | 6a3cb5fb206542251d7ca492 | | MagicDec | 101.75 | Not recorded | 6a3cb5fb206542251d7ca492 | | ClassicSD | 96.79 | Not recorded | 6a3cb5fb206542251d7ca492 | | MagicDec | 90.54 | Not recorded | 6a3cb5fb206542251d7ca492 | | ClassicSD | 88.45 | Not recorded | 6a3cb5fb206542251d7ca492 | | Vanilla | 82.77 | Not recorded | 6a3cb5fb206542251d7ca492 | | Vanilla | 73.24 | Not recorded | 6a3cb5fb206542251d7ca492 | | Vanilla | 60.32 | Not recorded | 6a3cb5fb206542251d7ca492 | | Vanilla | 52.17 | Not recorded | 6a3cb5fb206542251d7ca492 | ### Dustin Throughput: 486.5. Result ID: 6a3cda71fbc373592c517f77. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### Dustin Throughput: 397.19. Result ID: 6a3cda71fbc373592c517f6f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### Dustin Throughput: 379.82. Result ID: 6a3cda71fbc373592c517f67. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### Dustin Throughput: 351.17. Result ID: 6a3cda71fbc373592c51801b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch Size":"16","Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### MagicDec Throughput: 296.98. Result ID: 6a3cda71fbc373592c517f76. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### Dustin Throughput: 280.57. Result ID: 6a3cda71fbc373592c517f73. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### ClassicSD Throughput: 270.66. Result ID: 6a3cda71fbc373592c517f75. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### Dustin Throughput: 264.58. Result ID: 6a3cda71fbc373592c518017. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch Size":"8","Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### Vanilla Throughput: 249.36. Result ID: 6a3cda71fbc373592c517f74. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### Dustin Throughput: 221.47. Result ID: 6a3cda71fbc373592c517f6b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### MagicDec Throughput: 217.74. Result ID: 6a3cda71fbc373592c51801a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch Size":"16","Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### Vanilla Throughput: 206.22. Result ID: 6a3cda71fbc373592c517f70. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### MagicDec Throughput: 202.91. Result ID: 6a3cda71fbc373592c517f72. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### Dustin Throughput: 200.55. Result ID: 6a3cda71fbc373592c517f63. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### ClassicSD Throughput: 194.8. Result ID: 6a3cda71fbc373592c517f71. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### MagicDec Throughput: 171.8. Result ID: 6a3cda71fbc373592c518016. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch Size":"8","Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### ClassicSD Throughput: 168.33. Result ID: 6a3cda71fbc373592c518019. [Reporting paper]() Recorded evaluation configuration: {"Batch Size":"16","Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### ClassicSD Throughput: 149.5. Result ID: 6a3cda71fbc373592c518015. [Reporting paper]() Recorded evaluation configuration: {"Batch Size":"8","Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### MagicDec Throughput: 137.91. Result ID: 6a3cda71fbc373592c517f6e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### ClassicSD Throughput: 130.65. Result ID: 6a3cda71fbc373592c517f6d. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### MagicDec Throughput: 127.52. Result ID: 6a3cda71fbc373592c517f66. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### Vanilla Throughput: 124.43. Result ID: 6a3cda71fbc373592c518018. [Reporting paper]() Recorded evaluation configuration: {"Batch Size":"16","Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### ClassicSD Throughput: 122.97. Result ID: 6a3cda71fbc373592c517f65. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### Vanilla Throughput: 110.4. Result ID: 6a3cda71fbc373592c518014. [Reporting paper]() Recorded evaluation configuration: {"Batch Size":"8","Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### MagicDec Throughput: 101.75. Result ID: 6a3cda71fbc373592c517f6a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### ClassicSD Throughput: 96.79. Result ID: 6a3cda71fbc373592c517f69. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### MagicDec Throughput: 90.54. Result ID: 6a3cda71fbc373592c517f62. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### ClassicSD Throughput: 88.45. Result ID: 6a3cda71fbc373592c517f61. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### Vanilla Throughput: 82.77. Result ID: 6a3cda71fbc373592c517f6c. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### Vanilla Throughput: 73.24. Result ID: 6a3cda71fbc373592c517f68. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-14B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. ### Vanilla Throughput: 60.32. Result ID: 6a3cda71fbc373592c517f64. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Batch Size":"16"}. This record does not assert independent reproduction. ### Vanilla Throughput: 52.17. Result ID: 6a3cda71fbc373592c517f60. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-32B-Instruct","Batch Size":"8"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.