Paged Decode Attention Serving on Bimodal B4 Trace
1GPU Utilization RatioFlashInfer
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| FlashInferSelected route=FlashInfer baseline path, Page size=16, Hq=32, Hkv=8, d=128, Precision=FP162026.06 | 1 | 1 | 0 | 0 |
Loading the SOTA2 catalog…