Max token throughput on 64K/64K serving scenario 8xH100 node 1.0
9.3Max Throughput (K tok/s)gpt-oss-puzzle-88B
Evaluation results
| Method | Links | |
|---|---|---|
| gpt-oss-puzzle-88BKV precision=KV FP82026.02 | 9.3 | |
| HyperNova-60BKV precision=KV BF162026.02 | 6.9 | |
| gpt-oss-puzzle-88BKV precision=KV BF162026.02 | 6.5 | |
| gpt-oss-120BKV precision=KV FP82026.02 | 5.8 | |
| gpt-oss-120BKV precision=KV BF162026.02 | 4 |