LLM Pretraining
Benchmarks
DatasetSOTA methodMetricTrendResultsLast updated
13.3Perplexity
47
May 22, 2026
15.57Perplexity
16
May 22, 2026
0.861BPB
8
Feb 26, 2026
2.964Training Loss
8
Feb 26, 2026
18.288Perplexity
5
Jun 1, 2026
0.758BPB (Validation)
5
Apr 13, 2026
0.758Validation BPB
3
Apr 13, 2026
20Train step throughput
1
May 12, 2026