Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

Language Modeling research benchmarks – Page 16 · SOTA2 Research

Language Modeling

Benchmarks

Dataset NameSOTA MethodSort

Dataset NameSOTA methodMetricTrendResultsLast Updated

Billion-Word Benchmark (dev)

Mesh-TensorFlow (Transformer-Decoder)

23.5Word-Perplexity

11

Feb 26, 2026

One Billion Word Benchmark (val)

26.6Perplexity

11

May 7, 2026

One Billion Word (OBW) 100% train set (test)

39.15PPL

11

Feb 26, 2026

One Billion Word (OBW) 1% train set (test)

63.83PPL

11

Feb 26, 2026

Joint model with recurrent intent + slot label context

10.19PPL (LM)

11

Feb 26, 2026

37.26Normalized Accuracy

10

Jun 29, 2026

21-EuroParl (test)

16NGL

10

Jun 5, 2026

11.07Perplexity

10

Jun 2, 2026

Wikitext-2 raw v1

2.241Loss

10

Jun 1, 2026

WikiText-103 Medium (test)

35.37Perplexity (PPL)

10

Jun 1, 2026

WikiText-103 Medium (val)

33.76Perplexity (PPL)

10

Jun 1, 2026

40.83Validation Perplexity

10

Jun 1, 2026

MiniPile (train)

30.87Training Perplexity

10

Jun 1, 2026

WikiText-103 20w x 2048

9.603Perplexity (PPL)

10

May 28, 2026

5.29Perplexity (PPL)

10

May 28, 2026

Tülü 3 SFT Mixture

9.96Perplexity (PPL)

10

May 14, 2026

8-bit Adam + MoLS

16.22Score at 30K Tokens

10

May 8, 2026

LM1B GPT-2 small model size equivalent (test)

20.53Perplexity

10

Apr 15, 2026

3.156Loss

10

Apr 13, 2026

123.3Perplexity

10

Mar 24, 2026

218.8Perplexity

10

Mar 24, 2026

47.06Accuracy

10

Mar 11, 2026

LLaMA-350M pre-training (val)

ALIAS Adam version

2.707Validation Loss

10

Feb 26, 2026

GPT-2 124M held-out (test)

17.33Perplexity

10

Feb 26, 2026

General Knowledge Base Model Knowledge

-25.41GEN Loss

10

Feb 26, 2026

Page 16 of 37

Popular datasets

WikiText-103 WikiText-2 C4 LAMBADA WikiText

Follow for updates

Join SOTA2 Contact