Language Modeling on Wiki-40B (val) benchmark leaderboard · SOTA2 Research

Language Modeling on Wiki-40B (val)

14.955PerplexityFLASH-Quad

Evaluation Results

Method	Links
FLASH-Quad2022.02		14.955	96.1
FLASH-Quad2022.02		14.998	141.3
FLASH2022.02		15.109	62.5
Transformer++2022.02		15.254	261
FLASH2022.02		15.259	57
Transformer++2022.02		15.282	149.2
FLASH-Quad2022.02		15.305	71.3
Transformer+2022.02		15.478	261.9
Transformer++2022.02		15.489	96.6
Transformer+2022.02		15.515	149.3
FLASH2022.02		15.525	56.7
Transformer+2022.02		15.653	96.1
FLASH-Quad2022.02		15.879	59.5
FLASH2022.02		15.935	56.9
Transformer++2022.02		15.943	70.9
Transformer+2022.02		15.999	70.3
Combiner2022.02		16.119	77.9
Combiner2022.02		16.171	71.7
Combiner2022.02		16.344	71.8
FLASH2022.02		16.581	57.2
FLASH-Quad2022.02		16.633	54.1
Combiner2022.02		16.71	74.4
Transformer++2022.02		16.835	54.7
Transformer+2022.02		16.907	55.6
Combiner2022.02		17.338	75.5
Transformer2022.02		17.341	54
Transformer2022.02		17.731	149.1
Transformer2022.02		18.154	96.3
Transformer2022.02		18.254	260.7
Performer2022.02		18.52	1,386.9
Performer2022.02		18.547	1,518.9
Performer2022.02		18.987	1,526.7
Performer2022.02		18.989	1,439.7
Transformer2022.02		19.808	70.9
Performer2022.02		19.923	1,526.8