Neighborhood Attention Forward Pass on NATTEN Problem Set 2-dimensional (test)
99.3FMHA Match/Outperform Rate (Self Attn)Fused Neighborhood Attention
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Fused Neighborhood AttentionPrecision=FP32, Pass=Forward, Hardware=NVIDIA A100 GPU2024.03 | 99.3 | 96 | 74 | — | — | |
| Fused neighborhood attentionPrecision=FP16, Dimensionality=2D2024.03 | 98.6 | 100 | 100 | — | 92.4 | |
| GEMM-based Neighborhood AttentionPrecision=FP32, Pass=Forward, Hardware=NVIDIA A100 GPU2024.03 | 92.4 | 89.5 | — | 28.1 | — | |
| GEMM neighborhood attentionPrecision=FP16, Dimensionality=2D2024.03 | 59.3 | 84 | — | 0 | 29.8 | |
| Naive Neighborhood AttentionPrecision=FP32, Pass=Forward, Hardware=NVIDIA A100 GPU2024.03 | 52 | — | 11.7 | 5.4 | — | |
| Naive neighborhood attentionPrecision=FP16, Dimensionality=2D2024.03 | 32.9 | 16.4 | 0 | 0 | 15.8 |