GPU Inference Throughput on A100-SXM4-40GB
5,081Throughput (tok/s)DF-SSM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DF-SSMBatch=5122026.04 | 5,081 | — | |
| DF-SSMBatch=1282026.04 | 3,996 | — | |
| DF-SSMBatch=322026.04 | 1,963 | 4.1 | |
| DF-SSMBatch=82026.04 | 647 | 5.6 | |
| FP16 mamba-ssmBatch=322026.04 | 482 | — | |
| DF-SSMBatch=12026.04 | 299 | 21.4 | |
| FP16 mamba-ssmBatch=82026.04 | 116 | — | |
| FP16 mamba-ssmBatch=12026.04 | 14 | — |