Long-Context Generation on PG-19 (10K context length)
2.92Throughput Speedup (micro-averaged)SpecPV-4K
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SpecPV-4KBackbone=LLaMA3.1-8B-Instruct, Partial KV Budget=4K, KV Cache Offloading=disabled2025.12 | 2.92 | 3.29 | |
| SpecPV-2KBackbone=LLaMA3.1-8B-Instruct, Partial KV Budget=2K, KV Cache Offloading=disabled2025.12 | 2.88 | 3.3 | |
| SpecPV-8KBackbone=LLaMA3.1-8B-Instruct, Partial KV Budget=8K, KV Cache Offloading=disabled2025.12 | 2.53 | 3.34 | |
| EAGLE3-YARNBackbone=LLaMA3.1-8B-Instruct, KV Cache Offloading=disabled, Context Adaptation=YARN2025.12 | 2.49 | 3.38 | |
| TriForceBackbone=LLaMA3.1-8B-Instruct, Partial KV Budget=4K, KV Cache Offloading=disabled2025.12 | 1.72 | 2.62 | |
| TokenSwiftBackbone=LLaMA3.1-8B-Instruct, Partial KV Budget=4K, KV Cache Offloading=disabled2025.12 | 1.41 | 1.64 |