LLM Serving Performance on Qwen 7B Uniform distribution 32 model variants 2.5
0.86Throughput (req/s)Delta-CoMe
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Delta-CoMearrival rate (lambda)=1.02025.06 | 0.86 | 1.44 | |
| PRINMIXarrival rate (lambda)=1.02025.06 | 0.86 | 1.17 | |
| Delta-CoMearrival rate (lambda)=0.52025.06 | 0.42 | 0.81 | |
| PRINMIXarrival rate (lambda)=0.52025.06 | 0.42 | 0.79 | |
| Naivearrival rate (lambda)=1.02025.06 | 0.08 | 481.42 | |
| Naivearrival rate (lambda)=0.52025.06 | 0.07 | 253.93 |