Multi-Head Attention Performance on NVIDIA L40S GPU (FP8, hd=128, Causal Mask)
257.9Performance (TFLOPS)LLM-TL
Evaluation results
| Method | Links | |
|---|---|---|
| LLM-TLSequence Length=16k, Head Dimension=128, Datatype=FP8, GPU=L40S, Mask Type=causal2025.06 | 257.9 | |
| LLM-TLSequence Length=8k, Head Dimension=128, Datatype=FP8, GPU=L40S, Mask Type=causal2025.06 | 255.1 | |
| LLM-TLSequence Length=4k, Head Dimension=128, Datatype=FP8, GPU=L40S, Mask Type=causal2025.06 | 254.6 | |
| LLM-TLSequence Length=2k, Head Dimension=128, Datatype=FP8, GPU=L40S, Mask Type=causal2025.06 | 248.3 | |
| LLM-TLSequence Length=1k, Head Dimension=128, Datatype=FP8, GPU=L40S, Mask Type=causal2025.06 | 241.1 | |
| LLM-TLSequence Length=512, Head Dimension=128, Datatype=FP8, GPU=L40S, Mask Type=causal2025.06 | 224.8 |