Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

Language Modeling research benchmarks – Page 30 · SOTA2 Research

Language Modeling

Benchmarks

Dataset NameSOTA MethodSort

Dataset NameSOTA methodMetricTrendResultsLast Updated

LAMBADA 500 examples

23Accuracy (LAMBADA 500)

4

Apr 14, 2026

Public Pretraining Dataset OOD

1.606Loss

4

Apr 13, 2026

Public Pretraining Dataset (train)

1.33Loss

4

Apr 13, 2026

26.8Perplexity

4

Apr 6, 2026

dynActGLU(Swish)

1.336Loss

4

Mar 24, 2026

3.22PPL

4

Jun 23, 2026

57.56Accuracy (%)

4

Mar 13, 2026

FineWeb Nanochat 5M token (val)

AutoResearch-RL

2.681BPB (val)

4

Mar 10, 2026

OpenWebText ctx len. 1024 (val)

PGM 6 / 6 (dim. 1024)

21.43Perplexity

4

Feb 26, 2026

0.46Delta PPL (Llama2-7B)

4

Feb 26, 2026

94.22Accuracy

4

Feb 26, 2026

97.73Accuracy

4

Feb 26, 2026

76.92Accuracy

4

Feb 26, 2026

GPT-2 1,000 samples

Direct replacement

27.99PPL

4

Feb 26, 2026

ConceptMoE-A1B-24B

1.844Loss

4

Feb 26, 2026

OpenWebText GPT-2 124M NanoGPT (train)

2.955Loss

4

Feb 26, 2026

UIH Domain (test)

0.64L_inf

4

Feb 26, 2026

CF Domain (test)

0.33L_inf

4

Feb 26, 2026

Item-Text Domain (test)

0.86L_inf

4

Feb 26, 2026

General Domain (holdout test)

0.79L_inf

4

Feb 26, 2026

1.61Loss

4

Feb 26, 2026

65.629PPL

4

Feb 26, 2026

6.59GFLOPs/token

4

Feb 26, 2026

8.52PPL

4

Feb 26, 2026

9.78PPL

4

Feb 26, 2026

Page 30 of 37

Popular datasets

WikiText-103 WikiText-2 C4 LAMBADA WikiText

Follow for updates

Join SOTA2 Contact