Paged Decode Attention Serving on Bimodal B8 trace
1.299GPU RatioPersistentKV
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| PersistentKVSelected route=PersistentKV workqueue, split 20, Page size=16, Hq=32, Hkv=8, d=128, Precision=FP162026.06 | 1.299 | 1.265 | 6.104 | 4.201 |