Cluster routing and instance management on Cluster-level serving workload
52Energy SavingsDynamoLLM
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| DynamoLLMObjective=Min energy/cost/carbon w/ SLOs, Knobs=Routing, GPU freq, instance cnt, MoE support=×2026.05 | 52 | 38 |
Loading the SOTA2 catalog…