LLM Serving Performance on Qwen 7B (Zipf alpha=1.5, 32 variants)
0.87Throughput (req/s)Delta-CoMe
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Delta-CoMearrival rate (lambda)=1.02025.06 | 0.87 | 0.68 | |
| PRINMIXarrival rate (lambda)=1.02025.06 | 0.87 | 0.62 | |
| Delta-CoMearrival rate (lambda)=0.52025.06 | 0.42 | 0.55 | |
| PRINMIXarrival rate (lambda)=0.52025.06 | 0.42 | 0.52 | |
| Naivearrival rate (lambda)=0.52025.06 | 0.21 | 52.42 | |
| Naivearrival rate (lambda)=1.02025.06 | 0.18 | 198.48 |