LLM Inference on ShareGPT
58.33Throughput (RPS)EB(k^*)
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| EB(k^*)GPU=H200, Model=Gemma-3-1B-IT2026.05 | 58.33 | 10.6 | |
| v1GPU=H200, Model=Gemma-3-1B-IT2026.05 | 52.73 | — | |
| v0GPU=H200, Model=Gemma-3-1B-IT2026.05 | 51.24 | — | |
| EB(k^*)GPU=RTX PRO 6000, Model=Gemma-3-1B-IT2026.05 | 50.84 | 3.9 | |
| v0GPU=RTX PRO 6000, Model=Gemma-3-1B-IT2026.05 | 49.31 | — | |
| v1GPU=RTX PRO 6000, Model=Gemma-3-1B-IT2026.05 | 48.94 | — |