Neighborhood Attention Forward Pass on NATTEN Problem Set 3-dimensional (test)
98.6FMHA Attn Match RateFused Neighborhood Attention
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Fused Neighborhood AttentionPrecision=FP32, Pass=Forward, Hardware=NVIDIA A100 GPU2024.03 | 98.6 | 100 | — | — | — | |
| Fused neighborhood attentionPrecision=FP16, Dimensionality=3D2024.03 | 97.3 | 100 | — | — | 87 | |
| Naive Neighborhood AttentionPrecision=FP32, Pass=Forward, Hardware=NVIDIA A100 GPU2024.03 | 61.1 | — | — | 0 | — | |
| Naive neighborhood attentionPrecision=FP16, Dimensionality=3D2024.03 | 43.5 | — | — | 0 | 20.2 |