Prefill throughput on NOSA-8B
20,438.9Prefill Throughput (tok/s)Dense
Evaluation Results
| Method | Links | |
|---|---|---|
| DenseSequence Length=32K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.06 | 20,438.9 | |
| DenseSequence Length=64K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.06 | 13,701 | |
| SparDASequence Length=32K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 13,456 | |
| SparseSequence Length=32K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.06 | 12,896.5 | |
| InfiniGenSequence Length=32K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 12,838.5 | |
| SparseSequence Length=32K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 12,778.3 | |
| SparDASequence Length=64K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 12,386.1 | |
| SparDASequence Length=96K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 11,807.3 | |
| SparseSequence Length=64K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.06 | 11,418 | |
| InfiniGenSequence Length=64K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 11,372.4 | |
| SparseSequence Length=64K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 11,359.2 | |
| SparDASequence Length=128K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 11,332.7 | |
| SparseSequence Length=96K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.06 | 10,514.7 | |
| InfiniGenSequence Length=96K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 10,501.6 | |
| SparseSequence Length=96K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 10,448.5 | |
| DenseSequence Length=96K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.06 | 10,244.1 | |
| SparseSequence Length=128K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.06 | 9,832.5 | |
| SparseSequence Length=128K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 9,805.2 | |
| InfiniGenSequence Length=128K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=with offload2026.06 | 9,749.3 | |
| DenseSequence Length=128K, Batch Size=4, Hardware=NVIDIA H100 GPU, Offloading=no offload2026.06 | 8,118 |