Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

LLM Pretraining research benchmarks · SOTA2 Research

LLM Pretraining

Benchmarks

Dataset NameSOTA MethodSort

Dataset NameSOTA methodMetricTrendResultsLast Updated

Adam (Stable-SPAM)

13.3Perplexity

47

May 22, 2026

15.57Perplexity

16

May 22, 2026

FineWeb-Edu (val)

0.861BPB

8

Feb 26, 2026

FineWeb-Edu (train)

2.964Training Loss

8

Feb 26, 2026

18.288Perplexity

5

Jun 1, 2026

PleIAs SYNTH (val)

ALPHALAB + Opus 4.6

0.758BPB (Validation)

5

Apr 13, 2026

500-shard corpus (val)

0.758Validation BPB

3

Apr 13, 2026

20Train step throughput

1

May 12, 2026

Popular datasets

C4 FineWeb 500-shard corpus PleIAs SYNTH MLPerf C4

Follow for updates

Join SOTA2 Contact