Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

Language Modeling research benchmarks – Page 21 · SOTA2 Research

Language Modeling

Benchmarks

Dataset NameSOTA MethodSort

Dataset NameSOTA methodMetricTrendResultsLast Updated

DeepSeekMoE 145B

1.876Loss

8

Feb 26, 2026

wiki40b Vietnamese (test)

1.057BPC

8

Feb 26, 2026

wiki40b Hebrew (test)

1.27BPC

8

Feb 26, 2026

wiki40b Finnish (test)

0.937BPC

8

Feb 26, 2026

cc-100 English (test)

1.212BPC

8

Feb 26, 2026

wiki40b English (test)

1.077BPC

8

Feb 26, 2026

text8 English (test)

1.133BPC

8

Feb 26, 2026

Shakespeare (non-i.i.d.)

45.24Test Error

8

Feb 26, 2026

Shakespeare (i.i.d.)

44.52Test Error

8

Feb 26, 2026

OpSub + TCS (S)

254.37Test Perplexity

8

Feb 26, 2026

OpSub + OpSub-EMT

55.56Test Perplexity

8

Feb 26, 2026

Toronto Book Corpus (TBC) (test)

Staged Training

10.72PPL

8

Feb 26, 2026

OpenWebText 1 (val)

14.66Validation Perplexity

8

Feb 26, 2026

SlimPajama large-scale (train)

2.129L(ψ)

8

Feb 26, 2026

47.4CulturaX Score (ar)

8

Feb 26, 2026

MADLAD-400 (val)

Branch-Train-Mix

21.1PPL (ar)

8

Feb 26, 2026

2.253Text NLL (Validation)

7

Jul 9, 2026

FineWeb-Edu T=4096

fib / stagger W=12

41.45Perplexity

7

Jun 30, 2026

FineWeb-Edu T=2048

fib / stagger W=12

41.65Perplexity

7

Jun 30, 2026

FineWeb-Edu T=1024

33.37Perplexity

7

Jun 30, 2026

MLGym Language Modeling

3.5Loss

7

Jun 23, 2026

Turkish Language Modeling (val)

1.384BPC

7

Jun 18, 2026

Wiki2 and C4 (test)

5.69Perplexity (Wiki2)

7

Jun 9, 2026

Movie Reviews M (test)

63.39HOPPL

7

Jun 4, 2026

Recipe (R) (test)

19.9HOPPL

7

Jun 4, 2026

Page 21 of 37

Popular datasets

WikiText-103 WikiText-2 C4 LAMBADA WikiText

Follow for updates

Join SOTA2 Contact