Paged Decode Attention Serving on Uniform B8 trace
1.053GPU RatioPersistentKV
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| PersistentKVSelected route=PersistentKV workqueue, split 24, Page size=16, Hq=32, Hkv=8, d=128, Precision=FP162026.06 | 1.053 | 1.082 | 3.052 | 2.609 |