Matrix Multiplication on Synthetic Transformer Shapes Attention-Value Att ⊗ V
6.38Latency (µs)BWTA_Att
Evaluation results
| Method | Links | |
|---|---|---|
| BWTA_AttShape=Att: [128, 128], V: [128, 128]2026.04 | 6.38 | |
| BWTA_AttShape=Att: [512, 512], V: [512, 512]2026.04 | 9.77 | |
| cuBLASShape=Att: [128, 128], V: [128, 128]2026.04 | 38.68 | |
| FP16Shape=Att: [128, 128], V: [128, 128]2026.04 | 51.12 | |
| cuBLASShape=Att: [512, 512], V: [512, 512]2026.04 | 57.42 | |
| FP16Shape=Att: [512, 512], V: [512, 512]2026.04 | 145.9 | |
| BWTA_AttShape=Att: [2048, 2048], V: [2048, 2048]2026.04 | 201.6 | |
| cuBLASShape=Att: [2048, 2048], V: [2048, 2048]2026.04 | 611.9 | |
| FP16Shape=Att: [2048, 2048], V: [2048, 2048]2026.04 | 2,833 |