Multi-Head Attention on T4 GPU Synthetic Performance Benchmark
17.79Performance (TFLOPS)DeepSeek-V3 + Ours
Evaluation Results
| Method | Links | |
|---|---|---|
| DeepSeek-V3 + OursHead Dimension=128, Sequence Length=1k2025.06 | 17.79 | |
| cuDNNHead Dimension=64, Sequence Length=16k2025.06 | 13.76 |