Relative Pos. Attention on Llama2-7b (q=32, k=32) (1k)
114.85TFLOPS (Relative Pos. Attention)FlexAttention
Evaluation results
| Method | Links | |
|---|---|---|
| FlexAttentionImplementation=PyTorch FlexAttention2026.05 | 114.85 | |
| CuBridgeImplementation=CuBridge2026.05 | 105.8 | |
| Qimeng AttnImplementation=Qimeng Attention2026.05 | 71.05 | |
| TorchImplementation=Standard PyTorch2026.05 | 14.7 |