Image Description on COCO 2017 (val)
4,633Throughput (tok/s)HeteroServe
Evaluation results
| Method | Links | ||
|---|---|---|---|
| HeteroServeConfiguration=Cfg4: A100×4 (shared), GPUs=4, Cost=$64k2026.03 | 4,633 | 64 | |
| vLLMConfiguration=TP=4 (4×A100), GPUs=4, Cost=$64k2026.03 | 3,619 | 51 | |
| HeteroServeConfiguration=Cfg3: A100×2 (shared), GPUs=2, Cost=$32k2026.03 | 3,351 | 94 | |
| HeteroServeConfiguration=Cfg2: 4090×2+A100×2, GPUs=4, Cost=$38k2026.03 | 3,269 | 78 | |
| HeteroServeConfiguration=Cfg1: 4090+A100×2, GPUs=3, Cost=$35k2026.03 | 3,252 | 83 | |
| vLLMConfiguration=TP=2 (2×A100), GPUs=2, Cost=$32k2026.03 | 2,768 | 78 |