Tool-call Latency on Novel agentic workflow 6-turn
624Median per-turn Latency (ms)LayerScale
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LayerScalecaching=no response-cache hits2026.05 | 624 | — | |
| vLLMcaching=no response-cache hits2026.05 | 1,338 | 2.1 | |
| SGLangcaching=no response-cache hits2026.05 | 1,400 | 2.2 |