Neighborhood Attention Forward Pass on NATTEN Problem Set 1-dimensional (test)
100FMHA % Matched/Outperformed (NATTEN 1D)Fused neighborhood attention
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Fused neighborhood attentionPrecision=FP16, Dimensionality=1D2024.03 | 100 | 100 | 100 | — | 98.2 | |
| Fused Neighborhood AttentionPrecision=FP32, Pass=Forward, Hardware=NVIDIA A100 GPU2024.03 | 99.9 | 100 | 64.8 | — | — | |
| GEMM-based Neighborhood AttentionPrecision=FP32, Pass=Forward, Hardware=NVIDIA A100 GPU2024.03 | 98.4 | 99.9 | — | 37.7 | — | |
| GEMM neighborhood attentionPrecision=FP16, Dimensionality=1D2024.03 | 72 | 98.7 | — | 0 | 54.2 | |
| Naive Neighborhood AttentionPrecision=FP32, Pass=Forward, Hardware=NVIDIA A100 GPU2024.03 | 34.6 | — | 0 | 0 | — | |
| Naive neighborhood attentionPrecision=FP16, Dimensionality=1D2024.03 | 21.8 | 1.7 | 0 | 0 | 8.8 |