LLM Serving on MATH-plus (p90 latency)
157.83p90 Latency (ms/token)Oracle SJF
Evaluation Results
| Method | Links | |
|---|---|---|
| Oracle SJFModel=DeepSeek-R1, Workload Mode=Burst Mode2025.09 | 157.83 | |
| PARSModel=DeepSeek-R1, Workload Mode=Burst Mode2025.09 | 185.06 | |
| Oracle SJFModel=Llama, Workload Mode=Burst Mode2025.09 | 206.64 | |
| Listwise SJFModel=DeepSeek-R1, Workload Mode=Burst Mode2025.09 | 208.37 | |
| Pointwise SJFModel=DeepSeek-R1, Workload Mode=Burst Mode2025.09 | 231.73 | |
| Cross-Model PARSModel=DeepSeek-R1, Workload Mode=Burst Mode2025.09 | 232.14 | |
| PARSModel=Llama, Workload Mode=Burst Mode2025.09 | 249.51 | |
| Cross-Model PARSModel=Llama, Workload Mode=Burst Mode2025.09 | 281.86 | |
| Listwise SJFModel=Llama, Workload Mode=Burst Mode2025.09 | 282.4 | |
| Pointwise SJFModel=Llama, Workload Mode=Burst Mode2025.09 | 292.77 | |
| FCFSModel=Llama, Workload Mode=Burst Mode2025.09 | 632.7 | |
| FCFSModel=DeepSeek-R1, Workload Mode=Burst Mode2025.09 | 645.43 |