MoE VLM Serving on Kimi-VL-A3B BF16, 64 experts, top-8
11.81Throughput (tok/s)WiSP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| WiSPSetting=SIM, GPU budget=≈10.7 GiB, cap amount=82026.06 | 11.81 | 1.21 | |
| Naive baselineSetting=SIM, GPU budget=≈10.7 GiB, offload amount=242026.06 | 9.76 | — | |
| WiSPSetting=SIM, GPU budget=≈16.7 GiB, cap amount=82026.06 | 9.44 | 1.18 | |
| Naive baselineSetting=SIM, GPU budget=≈16.7 GiB, offload amount=202026.06 | 8.02 | — |