Grouped-Query Attention (GQA) on NVIDIA A100 GPU Synthetic Performance
154.7TFLOPSDeepSeek-V3 + Ours
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DeepSeek-V3 + OursHead Dimension=64, Sequence Length=2k, Causal Mask=true2025.06 | 154.7 | 35.16 | |
| DeepSeek-V3Head Dimension=64, Sequence Length=2k, Causal Mask=true2025.06 | 4.4 | — |