List your company
ResearchFormerly WizWand

Text Generation Inference Performance on Synthetic 1k Tokens

2,626Throughput (k tokens/s)SRM
-59.28637.861,3352,032.14May 9, 2026
Updated 5mo ago

Evaluation results

MethodLinks
SRMModel Dimension (dm)=512, Number of Layers (nl)=8, Inference Engine=Mojo/MAX, Hardware=1x H100 (96GB), Tokenizer Size=8k2026.052,6261,200
SRMModel Dimension (dm)=1024, Number of Layers (nl)=8, Inference Engine=Mojo/MAX, Hardware=1x H100 (96GB), Tokenizer Size=8k2026.051,203800
RWKVModel Dimension (dm)=512, Number of Layers (nl)=8, Inference Engine=Albatross, Hardware=1x H100 (96GB), Tokenizer Size=8k2026.0518450
TransformerModel Dimension (dm)=512, Number of Layers (nl)=8, Inference Engine=vLLM, Hardware=1x H100 (96GB), Tokenizer Size=8k2026.051015.32
MambaModel Dimension (dm)=512, Number of Layers (nl)=8, Inference Engine=vLLM, Hardware=1x H100 (96GB), Tokenizer Size=8k2026.054420