FineWeb
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
FineWeb Nemotron (val)
0.997R1
10
FineWeb 100M token (val)
12.11Perplexity
9
FineWeb-Edu (held-out)
4.733Final 50 Train Loss
8
FineWeb-Edu (val)
0.861BPB
8
FineWeb-Edu (train)
2.964Training Loss
8
FineWeb sample-10BT (train)
24.06Likelihood
7
FineWeb sample-10BT
805.77Likelihood
7
FineWeb sample-10BT
95.5MAUVE Score
7
FineWeb sample-10BT
0.893MAUVE
7
FineWeb-edu CC-MAIN-2024-10
81.9Recall@30
7
Fineweb-edu distillation 8B to 300M
62Spec. Accept %
7
Fineweb-edu 1.0 (test)
0.735Speculative Accept Rate
6
Fineweb-edu 1.0 (test)
2.32LM Loss
6
FineWeb Edu
25.37ARC-c
5
FineWeb
48.78Perplexity
5
FineWeb2
94.52Macro F1
5
FineWeb sample-10BT
593.46Likelihood
4
FineWeb sample-10BT
0.373MAUVE
4
FineWeb Nanochat 5M token (val)
2.681BPB (val)
4
Fineweb-Edu
0.392Accuracy
3
FineWeb 124M Transformer (val)
2.1Training Time to Loss 3.28 (min)
3
FineWeb in-distribution
19.73Perplexity (PPL)
2
FineWeb (val)
95.2Training Time (s)
2