Language Modeling on PG-19 500M parameters scale (test)
40.72PPLXFLASH
Evaluation results
| Method | Links | |
|---|---|---|
| FLASHContext Length=4096, Latency=234, Speedup=6.75x2022.02 | 40.72 | |
| FLASHContext Length=8192, Latency=250, Speedup=12.12x2022.02 | 41.07 | |
| FLASH-QuadContext Length=4096, Latency=371, Speedup=3.59x2022.02 | 41.46 | |
| FLASH-QuadContext Length=8192, Latency=560, Speedup=5.23x2022.02 | 41.68 | |
| FLASH-QuadContext Length=2048, Latency=273, Speedup=3.29x2022.02 | 42.01 | |
| FLASHContext Length=2048, Latency=237, Speedup=3.85x2022.02 | 42.17 | |
| Transformer+Context Length=4096, Latency=698, Speedup=1.00x2022.02 | 42.8 | |
| Transformer++Context Length=4096, Latency=7122022.02 | 43.13 | |
| Transformer+Context Length=2048, Latency=433, Speedup=1.00x2022.02 | 43.14 | |
| Transformer++Context Length=2048, Latency=4412022.02 | 43.18 | |
| Transformer++Context Length=8192, Latency=1272, Speedup=1.21x2022.02 | 43.26 | |
| Transformer+Context Length=8192, Latency=1292, Speedup=1.00x2022.02 | 43.27 | |
| FLASH-QuadContext Length=1024, Latency=231, Speedup=2.18x2022.02 | 43.4 | |
| CombinerContext Length=4096, Latency=3742022.02 | 43.99 | |
| FLASHContext Length=1024, Latency=234, Speedup=1.66x2022.02 | 44.06 | |
| CombinerContext Length=8192, Latency=4072022.02 | 44.12 | |
| Transformer+Context Length=1024, Latency=282, Speedup=1.00x2022.02 | 44.45 | |
| Transformer++Context Length=1024, Latency=2922022.02 | 44.47 | |
| CombinerContext Length=2048, Latency=3762022.02 | 44.68 | |
| CombinerContext Length=1024, Latency=3862022.02 | 46.04 |