Efficiency Evaluation on Efficiency Benchmark A40 4096 4096
160.32Latency (s)ChunkKV
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ChunkKVHardware=A40, Batch size=1, Input/output length=4096/40962025.02 | 160.32 | 41.3 | |
| ShotKVHardware=A40, Batch size=1, Input/output length=4096/40962025.02 | 162.85 | 41.12 | |
| SnapKVHardware=A40, Batch size=1, Input/output length=4096/40962025.02 | 163.45 | 40.51 | |
| FullKVHardware=A40, Batch size=1, Input/output length=4096/40962025.02 | 175.5 | 37.73 |