Agentic Serving on ShareGPT (vLLM, 32 concurrent sessions)
0.86P50 TTFT (Warm Rounds, r2-3) RatioUltraQuant
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| UltraQuantModel=MiniMax-M2.5, Tensor Parallelism (TP)=2, Hardware=AMD MI355X, Baseline comparison=FP8 KV2026.06 | 0.86 | 3.47 | 2.3 | 1.63 |