List your company
ResearchFormerly WizWand

Prefill throughput on NOSA-8B

20,438.9Prefill Throughput (tok/s)Dense
7,625.16410,951.80714,278.4517,605.093Jun 3, 2026
Updated 5mo ago

Evaluation results

MethodLinks
DenseSequence Length=32K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.0620,438.9
DenseSequence Length=64K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.0613,701
SparDASequence Length=32K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0613,456
SparseSequence Length=32K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.0612,896.5
InfiniGenSequence Length=32K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0612,838.5
SparseSequence Length=32K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0612,778.3
SparDASequence Length=64K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0612,386.1
SparDASequence Length=96K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0611,807.3
SparseSequence Length=64K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.0611,418
InfiniGenSequence Length=64K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0611,372.4
SparseSequence Length=64K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0611,359.2
SparDASequence Length=128K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0611,332.7
SparseSequence Length=96K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.0610,514.7
InfiniGenSequence Length=96K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0610,501.6
SparseSequence Length=96K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.0610,448.5
DenseSequence Length=96K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.0610,244.1
SparseSequence Length=128K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.069,832.5
SparseSequence Length=128K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.069,805.2
InfiniGenSequence Length=128K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.069,749.3
DenseSequence Length=128K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.068,118