Speculative Decoding on 20 Prompts across 4 Task Categories
6.55Mean Expected Tokens per Speculation StepSpecKV-acc.
Evaluation Results
| Method | Links | |
|---|---|---|
| SpecKV-acc.Compression=NF42026.05 | 6.55 | |
| SpecKV-acc.Compression=Overall2026.05 | 6.39 | |
| SpecKV-acc.Compression=INT82026.05 | 6.36 | |
| SpecKV-acc.Compression=FP162026.05 | 6.26 | |
| SpecKV-fastCompression=NF42026.05 | 5.93 | |
| Fixed-bestCompression=NF42026.05 | 5.92 | |
| SpecKV-fastCompression=INT82026.05 | 5.85 | |
| Fixed-bestCompression=INT82026.05 | 5.84 | |
| SpecKV-fastCompression=Overall2026.05 | 5.82 | |
| Fixed-bestCompression=Overall2026.05 | 5.81 | |
| SpecKV-fastCompression=FP162026.05 | 5.68 | |
| Fixed-bestCompression=FP162026.05 | 5.67 | |
| Task-oracleCompression=INT82026.05 | 5.43 | |
| Task-oracleCompression=NF42026.05 | 4.09 | |
| Task-oracleCompression=Overall2026.05 | 4.07 | |
| Fixed-4Compression=NF42026.05 | 3.78 | |
| Fixed-4Compression=INT82026.05 | 3.75 | |
| Fixed-4Compression=Overall2026.05 | 3.73 | |
| Fixed-4Compression=FP162026.05 | 3.67 | |
| Task-oracleCompression=FP162026.05 | 2.68 |