LLM Inference on Qwen3-8B workload (c=512) on RTX PRO 6000
80.3SLO Attainment (%)EB+
Evaluation results
| Method | Links | |
|---|---|---|
| EB+TTFT target=<10 s, TPOT target=< 100 ms2026.05 | 80.3 | |
| EB(k*)TTFT target=<10 s, TPOT target=< 100 ms2026.05 | 77.3 | |
| EB+TTFT target=<5 s, TPOT target=< 100 ms2026.05 | 48.4 | |
| EB(k*)TTFT target=<5 s, TPOT target=< 100 ms2026.05 | 6.2 | |
| v1TTFT target=<10 s, TPOT target=< 100 ms2026.05 | 5.8 | |
| v1TTFT target=<5 s, TPOT target=< 100 ms2026.05 | 5 | |
| v1TTFT target=<2 s, TPOT target=< 100 ms2026.05 | 4.9 | |
| EB+TTFT target=<10 s, TPOT target=< 50 ms2026.05 | 1.3 | |
| EB+TTFT target=<2 s, TPOT target=< 100 ms2026.05 | 1.2 | |
| EB+TTFT target=<5 s, TPOT target=< 50 ms2026.05 | 1.1 | |
| v1TTFT target=<2 s, TPOT target=< 50 ms2026.05 | 0.6 | |
| v1TTFT target=<5 s, TPOT target=< 50 ms2026.05 | 0.6 | |
| v1TTFT target=<10 s, TPOT target=< 50 ms2026.05 | 0.6 | |
| EB(k*)TTFT target=<10 s, TPOT target=< 50 ms2026.05 | 0.5 | |
| EB(k*)TTFT target=<2 s, TPOT target=< 50 ms2026.05 | 0 | |
| EB+TTFT target=<2 s, TPOT target=< 50 ms2026.05 | 0 | |
| EB(k*)TTFT target=<5 s, TPOT target=< 50 ms2026.05 | 0 | |
| EB(k*)TTFT target=<2 s, TPOT target=< 100 ms2026.05 | 0 |