Long-context generation efficiency on PG-19
407.96ThroughputDustin
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DustinBackbone=Llama-3.3-70B-Instruct, Batch Size=16, Context Length=8K2026.06 | 407.96 | 2.9 | 5.27 | |
| DustinBackbone=Qwen2.5-72B-Instruct, Batch Size=16, Context Length=8K2026.06 | 362.5 | 2.2 | 4.76 | |
| DustinBackbone=Qwen2.5-72B-Instruct, Batch Size=16, Context Length=16K2026.06 | 324.86 | 2.2 | 7.06 | |
| DustinBackbone=Llama-3.3-70B-Instruct, Batch Size=16, Context Length=16K2026.06 | 291.87 | 2.8 | 6.25 | |
| DustinBackbone=Qwen2.5-72B-Instruct, Batch Size=16, Context Length=24K2026.06 | 276.34 | 2.2 | 8.39 | |
| DustinBackbone=Llama-3.3-70B-Instruct, Batch Size=8, Context Length=8K2026.06 | 237.72 | 2.9 | 4.03 | |
| DustinBackbone=Qwen2.5-72B-Instruct, Batch Size=16, Context Length=32K2026.06 | 235.81 | 2.2 | 9.17 | |
| DustinBackbone=Llama-3.3-70B-Instruct, Batch Size=16, Context Length=24K2026.06 | 226.85 | 2.8 | 6.78 | |
| DustinBackbone=Llama-3.3-70B-Instruct, Batch Size=8, Context Length=16K2026.06 | 205.66 | 2.7 | 5.21 | |
| DustinBackbone=Llama-3.3-70B-Instruct, Batch Size=16, Context Length=32K2026.06 | 186.69 | 2.9 | 7.18 | |
| DustinBackbone=Llama-3.3-70B-Instruct, Batch Size=8, Context Length=24K2026.06 | 175.43 | 2.9 | 5.94 | |
| DustinBackbone=Qwen2.5-72B-Instruct, Batch Size=8, Context Length=8K2026.06 | 174.86 | 2.2 | 3.01 | |
| DustinBackbone=Qwen2.5-72B-Instruct, Batch Size=8, Context Length=16K2026.06 | 172.37 | 2.3 | 4.45 | |
| MagicDecBackbone=Llama-3.3-70B-Instruct, Batch Size=16, Context Length=8K2026.06 | 162.47 | 2.7 | 2.1 | |
| DustinBackbone=Qwen2.5-72B-Instruct, Batch Size=8, Context Length=24K2026.06 | 162.12 | 2.2 | 5.58 | |
| DustinBackbone=Qwen2.5-72B-Instruct, Batch Size=8, Context Length=32K2026.06 | 153.67 | 2.2 | 6.61 | |
| ClassicSDBackbone=Llama-3.3-70B-Instruct, Batch Size=16, Context Length=8K2026.06 | 151.33 | 2.7 | 1.96 | |
| DustinBackbone=Llama-3.3-70B-Instruct, Batch Size=8, Context Length=32K2026.06 | 148.2 | 2.8 | 6.28 | |
| ClassicSDBackbone=Qwen2.5-72B-Instruct, Batch Size=16, Context Length=8K2026.06 | 145.4 | 2.3 | 1.91 | |
| MagicDecBackbone=Qwen2.5-72B-Instruct, Batch Size=16, Context Length=8K2026.06 | 143.67 | 2.1 | 1.88 | |
| MagicDecBackbone=Llama-3.3-70B-Instruct, Batch Size=8, Context Length=8K2026.06 | 122.8 | 2.7 | 2.08 | |
| ClassicSDBackbone=Llama-3.3-70B-Instruct, Batch Size=8, Context Length=8K2026.06 | 115.91 | 2.7 | 1.96 | |
| ClassicSDBackbone=Qwen2.5-72B-Instruct, Batch Size=8, Context Length=8K2026.06 | 101.82 | 2.2 | 1.75 | |
| MagicDecBackbone=Qwen2.5-72B-Instruct, Batch Size=8, Context Length=8K2026.06 | 98.9 | 2.1 | 1.7 | |
| MagicDecBackbone=Llama-3.3-70B-Instruct, Batch Size=16, Context Length=16K2026.06 | 98.79 | 2.6 | 2.12 | |
| MagicDecBackbone=Qwen2.5-72B-Instruct, Batch Size=16, Context Length=16K2026.06 | 89.27 | 2.1 | 1.94 | |
| ClassicSDBackbone=Qwen2.5-72B-Instruct, Batch Size=16, Context Length=16K2026.06 | 89.26 | 2.2 | 1.94 | |
| ClassicSDBackbone=Llama-3.3-70B-Instruct, Batch Size=16, Context Length=16K2026.06 | 87.68 | 2.6 | 1.88 | |
| MagicDecBackbone=Llama-3.3-70B-Instruct, Batch Size=8, Context Length=16K2026.06 | 79.28 | 2.5 | 2.01 | |
| VanillaBackbone=Llama-3.3-70B-Instruct, Batch Size=16, Context Length=8K2026.06 | 77.38 | — | 1 | |
| VanillaBackbone=Qwen2.5-72B-Instruct, Batch Size=16, Context Length=8K2026.06 | 76.23 | — | 1 | |
| ClassicSDBackbone=Llama-3.3-70B-Instruct, Batch Size=8, Context Length=16K2026.06 | 75.31 | 2.6 | 1.91 | |
| MagicDecBackbone=Llama-3.3-70B-Instruct, Batch Size=16, Context Length=24K2026.06 | 71.2 | 2.8 | 2.13 | |
| ClassicSDBackbone=Qwen2.5-72B-Instruct, Batch Size=8, Context Length=16K2026.06 | 67.66 | 2.2 | 1.75 | |
| MagicDecBackbone=Qwen2.5-72B-Instruct, Batch Size=8, Context Length=16K2026.06 | 65.67 | 2 | 1.69 | |
| MagicDecBackbone=Qwen2.5-72B-Instruct, Batch Size=16, Context Length=24K2026.06 | 64.63 | 2.1 | 1.96 | |
| ClassicSDBackbone=Qwen2.5-72B-Instruct, Batch Size=16, Context Length=24K2026.06 | 63 | 2.2 | 1.91 | |
| ClassicSDBackbone=Llama-3.3-70B-Instruct, Batch Size=16, Context Length=24K2026.06 | 61.73 | 2.8 | 1.85 | |
| MagicDecBackbone=Llama-3.3-70B-Instruct, Batch Size=8, Context Length=24K2026.06 | 60.67 | 2.7 | 2.06 | |
| VanillaBackbone=Llama-3.3-70B-Instruct, Batch Size=8, Context Length=8K2026.06 | 59.01 | — | 1 | |
| VanillaBackbone=Qwen2.5-72B-Instruct, Batch Size=8, Context Length=8K2026.06 | 58.06 | — | 1 | |
| MagicDecBackbone=Llama-3.3-70B-Instruct, Batch Size=16, Context Length=32K2026.06 | 55.47 | 2.8 | 2.13 | |
| ClassicSDBackbone=Llama-3.3-70B-Instruct, Batch Size=8, Context Length=24K2026.06 | 55.14 | 2.8 | 1.87 | |
| ClassicSDBackbone=Qwen2.5-72B-Instruct, Batch Size=8, Context Length=24K2026.06 | 50.47 | 2.2 | 1.74 | |
| MagicDecBackbone=Qwen2.5-72B-Instruct, Batch Size=16, Context Length=32K2026.06 | 50.38 | 2 | 1.96 | |
| MagicDecBackbone=Qwen2.5-72B-Instruct, Batch Size=8, Context Length=24K2026.06 | 49.91 | 2.1 | 1.72 | |
| ClassicSDBackbone=Qwen2.5-72B-Instruct, Batch Size=16, Context Length=32K2026.06 | 48.68 | 2.1 | 1.89 | |
| MagicDecBackbone=Llama-3.3-70B-Instruct, Batch Size=8, Context Length=32K2026.06 | 48.48 | 2.8 | 2.05 | |
| ClassicSDBackbone=Llama-3.3-70B-Instruct, Batch Size=16, Context Length=32K2026.06 | 47.42 | 2.9 | 1.82 | |
| VanillaBackbone=Llama-3.3-70B-Instruct, Batch Size=16, Context Length=16K2026.06 | 46.69 | — | 1 | |
| VanillaBackbone=Qwen2.5-72B-Instruct, Batch Size=16, Context Length=16K2026.06 | 46.03 | — | 1 | |
| ClassicSDBackbone=Llama-3.3-70B-Instruct, Batch Size=8, Context Length=32K2026.06 | 43.41 | 2.9 | 1.84 | |
| ClassicSDBackbone=Qwen2.5-72B-Instruct, Batch Size=8, Context Length=32K2026.06 | 40.83 | 2.2 | 1.76 | |
| MagicDecBackbone=Qwen2.5-72B-Instruct, Batch Size=8, Context Length=32K2026.06 | 39.63 | 2 | 1.7 | |
| VanillaBackbone=Llama-3.3-70B-Instruct, Batch Size=8, Context Length=16K2026.06 | 39.44 | — | 1 | |
| VanillaBackbone=Qwen2.5-72B-Instruct, Batch Size=8, Context Length=16K2026.06 | 38.75 | — | 1 | |
| VanillaBackbone=Llama-3.3-70B-Instruct, Batch Size=16, Context Length=24K2026.06 | 33.45 | — | 1 | |
| VanillaBackbone=Qwen2.5-72B-Instruct, Batch Size=16, Context Length=24K2026.06 | 32.95 | — | 1 | |
| VanillaBackbone=Llama-3.3-70B-Instruct, Batch Size=8, Context Length=24K2026.06 | 29.52 | — | 1 | |
| VanillaBackbone=Qwen2.5-72B-Instruct, Batch Size=8, Context Length=24K2026.06 | 29.05 | — | 1 | |
| VanillaBackbone=Llama-3.3-70B-Instruct, Batch Size=16, Context Length=32K2026.06 | 26 | — | 1 | |
| VanillaBackbone=Qwen2.5-72B-Instruct, Batch Size=16, Context Length=32K2026.06 | 25.7 | — | 1 | |
| VanillaBackbone=Llama-3.3-70B-Instruct, Batch Size=8, Context Length=32K2026.06 | 23.61 | — | 1 | |
| VanillaBackbone=Qwen2.5-72B-Instruct, Batch Size=8, Context Length=32K2026.06 | 23.26 | — | 1 |