Efficiency Benchmarking on DeepSeek-R1-Distill-Llama-8B 16K Generation Length
8Max Batch SizeSnapKV
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| SnapKVBudget=Fixed - 10242026.06 | 8 | 222.16 | 5.6 | 589.98 | |
| SnapKVBudget=Fixed - 15362026.06 | 8 | 186.47 | 4.7 | 702.93 | |
| SnapKVBudget=Fixed - 20482026.06 | 8 | 162.03 | 4.09 | 808.95 | |
| R-KVBudget=Fixed - 10242026.06 | 8 | 218.85 | 5.52 | 598.92 | |
| R-KVBudget=Fixed - 15362026.06 | 8 | 183.81 | 4.63 | 713.07 | |
| R-KVBudget=Fixed - 20482026.06 | 8 | 160.05 | 4.04 | 818.93 | |
| ReasonAllocBudget=Dynamic - 10242026.06 | 8 | 218.82 | 5.52 | 598.99 | |
| ReasonAllocBudget=Dynamic - 15362026.06 | 8 | 184.33 | 4.65 | 711.07 | |
| ReasonAllocBudget=Dynamic - 20482026.06 | 8 | 159.7 | 4.03 | 820.73 | |
| FullKVBudget=FullKV2026.06 | 2 | 39.66 | 1 | 826.27 |