LLM Serving on Alpaca (Average Latency)
111.46Average Latency (ms/token)Oracle SJF
Evaluation Results
| Method | Links | |
|---|---|---|
| Oracle SJFModel=Llama2025.09 | 111.46 | |
| PARSModel=Llama2025.09 | 124.92 | |
| Oracle SJFModel=DeepSeek-R12025.09 | 137.32 | |
| Cross-Model PARSModel=Llama2025.09 | 138.53 | |
| PARSModel=DeepSeek-R12025.09 | 148.97 | |
| Pointwise SJFModel=Llama2025.09 | 152.3 | |
| Listwise SJFModel=Llama2025.09 | 157.07 | |
| Cross-Model PARSModel=DeepSeek-R12025.09 | 178.07 | |
| Listwise SJFModel=DeepSeek-R12025.09 | 187.41 | |
| Pointwise SJFModel=DeepSeek-R12025.09 | 283.39 | |
| FCFSModel=DeepSeek-R12025.09 | 372.03 | |
| FCFSModel=Llama2025.09 | 962.53 |