Long-range sequence modeling on ListOps Long Range Arena (LRA) 2K (test)
37.9AccuracyEfficient Attention + GFSA
Evaluation results
| Method | Links | |
|---|---|---|
| Efficient Attention + GFSARuntime (s per 1,000 steps)=53.8, GPU usage (GB)=0.672023.12 | 37.9 | |
| Transformer+GFSARuntime (s per 1,000 steps)=635.8, GPU usage (GB)=10.872023.12 | 37.6 | |
| LinformerRuntime (s per 1,000 steps)=63.4, GPU usage (GB)=1.732023.12 | 37.3 | |
| YOSO-ERuntime (s per 1,000 steps)=85.7, GPU usage (GB)=0.372023.12 | 37.3 | |
| TransformerRuntime (s per 1,000 steps)=198.3, GPU usage (GB)=5.52023.12 | 37.1 | |
| Efficient AttentionRuntime (s per 1,000 steps)=49.2, GPU usage (GB)=0.572023.12 | 36.9 |