Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

Pre-training corpus dataset and benchmarks · SOTA2 Research

Research Datasets

Pre-training corpus

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Language Modeling

Pre-training corpus (train)

15.71Perplexity

20

Language Modeling

Pre-training corpus

1.577Loss

9

Next token prediction

Pre-training corpus (train)

66.4Token Accuracy

9

Language Modeling

1.3B 26B-token pre-training corpus (val)

2.077Validation Cross-Entropy

3

Popular tasks

Language Modeling Next-token prediction

Follow for updates

Join SOTA2 Contact