ResearchBenchmarksLanguage Modeling on Pile Non-AR tokensFollow33.95PerplexityAttention33.827234.656135.48536.3139Feb 16, 2024Evaluation ResultsMethodMethodLinksPerplexityAttentionArchitecture=AttentionArchitecture=Attention2024.0233.95ReBasedArchitecture=ReBasedArchitecture=ReBased2024.0236.73BasedArchitecture=BasedArchitecture=Based2024.0237.02