Long-Context Generation on PG-19 16K context length
486.5ThroughputDustin
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DustinModel=Qwen2.5-7B-Instruct, Batch Size=162026.06 | 486.5 | 2.3 | 1.95 | |
| DustinModel=Qwen2.5-14B-Instruct, Batch Size=162026.06 | 397.19 | 2.1 | 4.8 | |
| DustinModel=Qwen2.5-32B-Instruct, Batch Size=162026.06 | 379.82 | 2.4 | 6.3 | |
| DustinBatch Size=16, Backbone=Llama-3.1-8B-Instruct2026.06 | 351.17 | 2.7 | 2.82 | |
| MagicDecModel=Qwen2.5-7B-Instruct, Batch Size=162026.06 | 296.98 | 2.3 | 1.19 | |
| DustinModel=Qwen2.5-7B-Instruct, Batch Size=82026.06 | 280.57 | 2.4 | 1.36 | |
| ClassicSDModel=Qwen2.5-7B-Instruct, Batch Size=162026.06 | 270.66 | 2.4 | 1.09 | |
| DustinBatch Size=8, Backbone=Llama-3.1-8B-Instruct2026.06 | 264.58 | 2.7 | 2.4 | |
| VanillaModel=Qwen2.5-7B-Instruct, Batch Size=162026.06 | 249.36 | — | 1 | |
| DustinModel=Qwen2.5-14B-Instruct, Batch Size=82026.06 | 221.47 | 2.1 | 3.02 | |
| MagicDecBatch Size=16, Backbone=Llama-3.1-8B-Instruct2026.06 | 217.74 | 2.6 | 1.75 | |
| VanillaModel=Qwen2.5-7B-Instruct, Batch Size=82026.06 | 206.22 | — | 1 | |
| MagicDecModel=Qwen2.5-7B-Instruct, Batch Size=82026.06 | 202.91 | 2.3 | 0.98 | |
| DustinModel=Qwen2.5-32B-Instruct, Batch Size=82026.06 | 200.55 | 2.3 | 3.84 | |
| ClassicSDModel=Qwen2.5-7B-Instruct, Batch Size=82026.06 | 194.8 | 2.4 | 0.94 | |
| MagicDecBatch Size=8, Backbone=Llama-3.1-8B-Instruct2026.06 | 171.8 | 2.6 | 1.56 | |
| ClassicSDBatch Size=16, Backbone=Llama-3.1-8B-Instruct2026.06 | 168.33 | 2.7 | 1.35 | |
| ClassicSDBatch Size=8, Backbone=Llama-3.1-8B-Instruct2026.06 | 149.5 | 2.7 | 1.35 | |
| MagicDecModel=Qwen2.5-14B-Instruct, Batch Size=162026.06 | 137.91 | 2 | 1.67 | |
| ClassicSDModel=Qwen2.5-14B-Instruct, Batch Size=162026.06 | 130.65 | 2 | 1.58 | |
| MagicDecModel=Qwen2.5-32B-Instruct, Batch Size=162026.06 | 127.52 | 2.3 | 2.11 | |
| VanillaBatch Size=16, Backbone=Llama-3.1-8B-Instruct2026.06 | 124.43 | — | 1 | |
| ClassicSDModel=Qwen2.5-32B-Instruct, Batch Size=162026.06 | 122.97 | 2.4 | 2.04 | |
| VanillaBatch Size=8, Backbone=Llama-3.1-8B-Instruct2026.06 | 110.4 | — | 1 | |
| MagicDecModel=Qwen2.5-14B-Instruct, Batch Size=82026.06 | 101.75 | 2 | 1.39 | |
| ClassicSDModel=Qwen2.5-14B-Instruct, Batch Size=82026.06 | 96.79 | 2 | 1.32 | |
| MagicDecModel=Qwen2.5-32B-Instruct, Batch Size=82026.06 | 90.54 | 2.1 | 1.74 | |
| ClassicSDModel=Qwen2.5-32B-Instruct, Batch Size=82026.06 | 88.45 | 2.3 | 1.7 | |
| VanillaModel=Qwen2.5-14B-Instruct, Batch Size=162026.06 | 82.77 | — | 1 | |
| VanillaModel=Qwen2.5-14B-Instruct, Batch Size=82026.06 | 73.24 | — | 1 | |
| VanillaModel=Qwen2.5-32B-Instruct, Batch Size=162026.06 | 60.32 | — | 1 | |
| VanillaModel=Qwen2.5-32B-Instruct, Batch Size=82026.06 | 52.17 | — | 1 |