Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

Language Modeling research benchmarks – Page 36 · SOTA2 Research

Language Modeling

Benchmarks

Dataset NameSOTA MethodSort

Dataset NameSOTA methodMetricTrendResultsLast Updated

Clustered Pre-training Data Cluster 1

17.731Perplexity (PPL)

2

Mar 20, 2026

Clustered Pre-training Data Cluster 0

19.819Perplexity (PPL)

2

Mar 20, 2026

Standard Language Modeling Benchmark

6.19Perplexity (PPL)

2

Mar 19, 2026

Mini-Qwen3 (val)

1.763Validation Loss

2

Mar 10, 2026

DS Mini Lite V2 (val)

2.469Validation Loss

2

Mar 10, 2026

Mini-GPT-OSS (val)

2.94Validation Loss

2

Mar 10, 2026

Language Modeling Benchmark Suite

Language model SOTA

1.04DM Mathematics Score

2

Feb 27, 2026

17.283Accuracy

2

Feb 26, 2026

20.54Perplexity

2

Jun 2, 2026

OWT L=10240 (test)

23.4Gen. PPL

2

Feb 26, 2026

8.9Perplexity

2

Feb 26, 2026

0.004MAE

2

Feb 26, 2026

20B token pretraining corpus

2.756PT Train Loss

2

Feb 26, 2026

79.87PPL

2

Feb 26, 2026

Turkish Legal Subdomains (Evaluation)

4.986CL

2

Feb 26, 2026

Transcriptions of White House Speeches En (test)

0.902BPC (df)

2

Feb 26, 2026

CNN news En (test)

0.85BPC (df)

2

Feb 26, 2026

Twitter posts En (test)

1.789BPC (df)

2

Feb 26, 2026

3.9Training Throughput

2

Feb 26, 2026

10.89Perplexity

2

Feb 26, 2026

14.2Perplexity

2

Feb 26, 2026

107.95Perplexity

2

Feb 26, 2026

Word-LM (train)

40.43Perplexity

2

Feb 26, 2026

1.3Bits Per Character

2

Feb 26, 2026

Char-LM (train)

1.066BPC

2

Feb 26, 2026

Page 36 of 37

Popular datasets

WikiText-103 WikiText-2 C4 LAMBADA WikiText

Follow for updates

Join SOTA2 Contact