Efficiency analysis on Long-Range Arena (LRA)
99.6Steps per secondPerformer
Evaluation results
| Method | Links | |
|---|---|---|
| PerformerPhase=Inference, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 99.6 | |
| FlowformerPhase=Inference, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 98.83 | |
| Local AttentionPhase=Inference, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 98.28 | |
| Linear Trans.Phase=Inference, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 97.33 | |
| LinformerPhase=Inference, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 96.84 | |
| PerformerPhase=Inference, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 96.8 | |
| LinformerPhase=Inference, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 96.56 | |
| PerformerPhase=Inference, Sequence Length=3K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 96.52 | |
| Local AttentionPhase=Inference, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 96.51 | |
| CosformerPhase=Inference, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 96.46 | |
| PerformerPhase=Inference, Sequence Length=4K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 96.42 | |
| FlowformerPhase=Inference, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 96.21 | |
| Linear Trans.Phase=Inference, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 96.14 | |
| FlowformerPhase=Inference, Sequence Length=4K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 95.82 | |
| FlowformerPhase=Inference, Sequence Length=3K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 95.65 | |
| Local AttentionPhase=Inference, Sequence Length=4K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 95.6 | |
| CosformerPhase=Inference, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 95.58 | |
| CosformerPhase=Inference, Sequence Length=3K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 95.19 | |
| LinformerPhase=Inference, Sequence Length=3K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 94.74 | |
| CosformerPhase=Inference, Sequence Length=4K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 94.69 | |
| Local AttentionPhase=Inference, Sequence Length=3K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 94.6 | |
| Linear Trans.Phase=Inference, Sequence Length=3K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 94.03 | |
| Linear Trans.Phase=Inference, Sequence Length=4K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 93.69 | |
| LinformerPhase=Inference, Sequence Length=4K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 93.59 | |
| Sinkhorn Trans.Phase=Inference, Sequence Length=3K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 92.72 | |
| Sinkhorn Trans.Phase=Inference, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 92.21 | |
| Sinkhorn Trans.Phase=Inference, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 91.42 | |
| BigBirdPhase=Inference, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 82.5 | |
| TransformerPhase=Inference, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 81.83 | |
| Sinkhorn Trans.Phase=Inference, Sequence Length=4K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 80.67 | |
| Sparse Trans.Phase=Inference, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 78.3 | |
| LongformerPhase=Inference, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 73.56 | |
| SynthesizerPhase=Inference, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 65.44 | |
| ReformerPhase=Inference, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 60.92 | |
| ReformerPhase=Inference, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 60.3 | |
| BigBirdPhase=Inference, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 54.12 | |
| FlowformerPhase=Train, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 49.76 | |
| Linear Trans.Phase=Train, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 48.78 | |
| Linear Trans.Phase=Train, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 48.66 | |
| PerformerPhase=Train, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 48.3 | |
| PerformerPhase=Train, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 47.34 | |
| FlowformerPhase=Train, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 47.18 | |
| Local AttentionPhase=Train, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 46.75 | |
| CosformerPhase=Train, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 46.5 | |
| ReformerPhase=Train, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 46.07 | |
| Sinkhorn Trans.Phase=Train, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 45.93 | |
| LinformerPhase=Train, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 45.57 | |
| CosformerPhase=Train, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 45.24 | |
| LinformerPhase=Train, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 44.11 | |
| Local AttentionPhase=Train, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 43.05 | |
| FlowformerPhase=Train, Sequence Length=3K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 41.93 | |
| Linear Trans.Phase=Train, Sequence Length=3K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 41.66 | |
| PerformerPhase=Train, Sequence Length=3K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 41 | |
| CosformerPhase=Train, Sequence Length=3K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 39.49 | |
| ReformerPhase=Inference, Sequence Length=3K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 39.37 | |
| BigBirdPhase=Inference, Sequence Length=3K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 37.83 | |
| LinformerPhase=Train, Sequence Length=3K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 37.28 | |
| FlowformerPhase=Train, Sequence Length=4K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 36.79 | |
| Sinkhorn Trans.Phase=Train, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 36.21 | |
| PerformerPhase=Train, Sequence Length=4K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 36.14 | |
| Linear Trans.Phase=Train, Sequence Length=4K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 35.44 | |
| Local AttentionPhase=Train, Sequence Length=3K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 35.42 | |
| CosformerPhase=Train, Sequence Length=4K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 35.09 | |
| LinformerPhase=Train, Sequence Length=4K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 31.58 | |
| Local AttentionPhase=Train, Sequence Length=4K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 30.34 | |
| BigBirdPhase=Inference, Sequence Length=4K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 29.34 | |
| Sinkhorn Trans.Phase=Train, Sequence Length=3K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 28.11 | |
| BigBirdPhase=Train, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 27.34 | |
| ReformerPhase=Inference, Sequence Length=4K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 26.98 | |
| TransformerPhase=Inference, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 25.26 | |
| Sinkhorn Trans.Phase=Train, Sequence Length=4K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 23.83 | |
| Sparse Trans.Phase=Inference, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 23.33 | |
| ReformerPhase=Train, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 22.93 | |
| TransformerPhase=Train, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 22.12 | |
| Sparse Trans.Phase=Train, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 21.74 | |
| BigBirdPhase=Train, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 16.95 | |
| ReformerPhase=Train, Sequence Length=3K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 14.34 | |
| LongformerPhase=Train, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 13.09 | |
| BigBirdPhase=Train, Sequence Length=3K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 12 | |
| ReformerPhase=Train, Sequence Length=4K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 9.56 | |
| BigBirdPhase=Train, Sequence Length=4K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 9.33 | |
| TransformerPhase=Train, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 7.5 | |
| Sparse Trans.Phase=Train, Sequence Length=2K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 7.3 | |
| SynthesizerPhase=Train, Sequence Length=1K, Hardware=2 NVIDIA 2080 Ti GPUs2022.02 | 5.16 |