LLM Inference on Qwen3-8B synthetic workload (mu_L=512, mu_O=256)
48,015Throughput (tok/s)EB+
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| EB+Hardware=2x H200, Concurrency (c)=2048, Allocation Strategy=DP=22026.05 | 48,015 | 6.33 | 99.71 | |
| v1Hardware=2x H200, Concurrency (c)=2048, Allocation Strategy=DP=22026.05 | 46,042 | 7.601 | 97.44 | |
| v1Hardware=2x H200, Concurrency (c)=512, Allocation Strategy=DP=22026.05 | 45,731 | 0.788 | 29.18 | |
| EB+Hardware=2x H200, Concurrency (c)=512, Allocation Strategy=DP=22026.05 | 45,539 | 0.779 | 29.44 | |
| vLLM DisaggHardware=2x H200, Concurrency (c)=512, Allocation Strategy=1P+1D disaggregation2026.05 | 39,649 | 3.608 | 22.81 | |
| EB+Hardware=2x H200, Concurrency (c)=64, Allocation Strategy=DP=22026.05 | 26,857 | 0.04 | 7.03 | |
| v1Hardware=2x H200, Concurrency (c)=64, Allocation Strategy=DP=22026.05 | 26,545 | 0.042 | 7.11 | |
| vLLM DisaggHardware=2x H200, Concurrency (c)=64, Allocation Strategy=1P+1D disaggregation2026.05 | 22,825 | 0.091 | 8.08 | |
| EB+Hardware=2x RTX PRO 6000, Concurrency (c)=512, Allocation Strategy=DP=22026.05 | 21,710 | 1.9 | 58.4 | |
| v1Hardware=2x RTX PRO 6000, Concurrency (c)=512, Allocation Strategy=DP=22026.05 | 21,526 | 1.9 | 58.9 | |
| EB+Hardware=2x RTX PRO 6000, Concurrency (c)=2048, Allocation Strategy=DP=22026.05 | 19,926 | 45.3 | 63.8 | |
| vLLM DisaggHardware=2x RTX PRO 6000, Concurrency (c)=512, Allocation Strategy=1P+1D disaggregation2026.05 | 16,467 | 6.219 | 58.64 | |
| v1Hardware=2x RTX PRO 6000, Concurrency (c)=2048, Allocation Strategy=DP=22026.05 | 16,228 | 45.9 | 150 | |
| EB+Hardware=2x RTX PRO 6000, Concurrency (c)=64, Allocation Strategy=DP=22026.05 | 8,691 | 0.084 | 22.1 | |
| vLLM DisaggHardware=2x RTX PRO 6000, Concurrency (c)=64, Allocation Strategy=1P+1D disaggregation2026.05 | 8,606 | 0.164 | 21.1 | |
| v1Hardware=2x RTX PRO 6000, Concurrency (c)=64, Allocation Strategy=DP=22026.05 | 8,529 | 0.087 | 22.3 |