Latency Evaluation on NSA Workload Head Dimension 128
0.67Latency (s)LLM-TL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LLM-TLSequence Length=512, Head Dimension=128, Hardware=A100 GPU2025.06 | 0.67 | 1.25 | |
| Naive NSASequence Length=512, Head Dimension=128, Hardware=A100 GPU2025.06 | 0.84 | — | |
| LLM-TLSequence Length=1k, Head Dimension=128, Hardware=A100 GPU2025.06 | 1.26 | 1.33 | |
| Naive NSASequence Length=1k, Head Dimension=128, Hardware=A100 GPU2025.06 | 1.68 | — | |
| LLM-TLSequence Length=2k, Head Dimension=128, Hardware=A100 GPU2025.06 | 2.59 | 1.29 | |
| Naive NSASequence Length=2k, Head Dimension=128, Hardware=A100 GPU2025.06 | 3.35 | — | |
| LLM-TLSequence Length=4k, Head Dimension=128, Hardware=A100 GPU2025.06 | 5.25 | 1.26 | |
| Naive NSASequence Length=4k, Head Dimension=128, Hardware=A100 GPU2025.06 | 6.61 | — | |
| LLM-TLSequence Length=8k, Head Dimension=128, Hardware=A100 GPU2025.06 | 10.59 | 1.26 | |
| Naive NSASequence Length=8k, Head Dimension=128, Hardware=A100 GPU2025.06 | 13.34 | — | |
| LLM-TLSequence Length=16k, Head Dimension=128, Hardware=A100 GPU2025.06 | 21.27 | 1.24 | |
| Naive NSASequence Length=16k, Head Dimension=128, Hardware=A100 GPU2025.06 | 26.29 | — |