Peak decode throughput on LLaDA 16B 16K context 2.0-mini
3,475Throughput (tokens/s)HERALD
Evaluation results
| Method | Links | ||
|---|---|---|---|
| HERALDBatch size (B)=1612026.06 | 3,475 | 2.13 | |
| DenseBatch size (B)=352026.06 | 1,629 | 1 | |
| MAGE-OffloadBatch size (B)=1282026.06 | 796 | 0.49 | |
| InfiniGenBatch size (B)=162026.06 | 555 | 0.34 |