guided-JSON extraction on public-synthetic compliance benchmark
3.6Throughput (req/s)vLLM + EAGLE-style
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| vLLM + EAGLE-styleGPUs=2×H1002026.05 | 3.6 | 2,067 | 0.154 | |
| vLLM + EAGLE-styleGPUs=1×H1002026.05 | 3.19 | 2,292 | 0.087 | |
| NIM baselineGPUs=2×H1002026.05 | 3.01 | 2,397 | 0.185 |