Text Generation Inference Performance on Synthetic 1k Tokens
2,626Throughput (k tokens/s)SRM
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| SRMModel Dimension (dm)=512, Number of Layers (nl)=8, Inference Engine=Mojo/MAX, Hardware=1x H100 (96GB), Tokenizer Size=8k2026.05 | 2,626 | 1,200 | |
| SRMModel Dimension (dm)=1024, Number of Layers (nl)=8, Inference Engine=Mojo/MAX, Hardware=1x H100 (96GB), Tokenizer Size=8k2026.05 | 1,203 | 800 | |
| RWKVModel Dimension (dm)=512, Number of Layers (nl)=8, Inference Engine=Albatross, Hardware=1x H100 (96GB), Tokenizer Size=8k2026.05 | 184 | 50 | |
| TransformerModel Dimension (dm)=512, Number of Layers (nl)=8, Inference Engine=vLLM, Hardware=1x H100 (96GB), Tokenizer Size=8k2026.05 | 101 | 5.32 | |
| MambaModel Dimension (dm)=512, Number of Layers (nl)=8, Inference Engine=vLLM, Hardware=1x H100 (96GB), Tokenizer Size=8k2026.05 | 44 | 20 |