MoE LLM Serving on Qwen3-30B-A3B (BF16, top-8 experts)
26.79Throughput (tokens/s)WiSP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| WiSPSetting=SIM, GPU budget=≈67 GiB, cap amount=1282026.06 | 26.79 | 1.01 | |
| Naive baselineSetting=SIM, GPU budget=≈67 GiB, offload amount=152026.06 | 26.53 | — | |
| WiSPSetting=SIM, GPU budget=≈44 GiB, cap amount=642026.06 | 25.75 | 1.95 | |
| WiSPSetting=SIM, GPU budget=≈32 GiB, cap amount=322026.06 | 18.51 | 1.8 | |
| WiSPSetting=SIM, GPU budget=≈23 GiB, cap amount=162026.06 | 13.79 | 1.54 | |
| Naive baselineSetting=SIM, GPU budget=≈44 GiB, offload amount=302026.06 | 13.21 | — | |
| Naive baselineSetting=SIM, GPU budget=≈32 GiB, offload amount=442026.06 | 10.27 | — | |
| Naive baselineSetting=SIM, GPU budget=≈23 GiB, offload amount=502026.06 | 8.96 | — |