Decode Throughput Generation on NVIDIA B200 GPU
6,742.39Throughput (tokens/s)YOCO (CLSA)
Evaluation Results
| Method | Links | |
|---|---|---|
| YOCO (CLSA)Context Length=8K2026.06 | 6,742.39 | |
| YOCO (CLSA)Context Length=16K2026.06 | 6,350.39 | |
| YOCO (Dense)Context Length=8K2026.06 | 5,516.5 | |
| YOCO (CLSA)Context Length=32K2026.06 | 5,461.33 | |
| TransformerContext Length=8K2026.06 | 4,762.79 | |
| YOCO (Dense)Context Length=16K2026.06 | 4,147.85 | |
| YOCO (CLSA)Context Length=64K2026.06 | 4,137.37 | |
| YOCO (CLSA)Context Length=128K2026.06 | 3,276.8 | |
| TransformerContext Length=16K2026.06 | 3,091.32 | |
| YOCO (Dense)Context Length=32K2026.06 | 2,677.12 | |
| TransformerContext Length=32K2026.06 | 1,761.72 | |
| YOCO (Dense)Context Length=64K2026.06 | 1,600 | |
| YOCO (Dense)Context Length=128K2026.06 | 960.94 | |
| TransformerContext Length=64K2026.06 | 948.15 | |
| TransformerContext Length=128K2026.06 | 431.16 |