Large Language Model Evaluation on 12-task evaluation suite composite (test)
49.6Reading Comprehension ScoreFineWeb-Edu
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| FineWeb-EduModel Architecture=MoE, Model Parameters=7B, Pre-training Token Budget=300B tokens2025.12 | 49.6 | 60.5 | 46.2 | 51.2 | 212 | |
| FineWebProModel Architecture=MoE, Model Parameters=7B, Pre-training Token Budget=300B tokens2025.12 | 49.3 | 60.7 | 45.9 | 51.4 | 212 | |
| FineWeb-MaskModel Architecture=MoE, Model Parameters=7B, Pre-training Token Budget=300B tokens2025.12 | 49.1 | 59.9 | 48.9 | 52.6 | 412 | |
| FineWeb-DCLMModel Architecture=MoE, Model Parameters=7B, Pre-training Token Budget=300B tokens2025.12 | 49 | 60.4 | 47.8 | 52.2 | 112 | |
| FineWeb-MaskModel Architecture=dense, Model Parameters=1.4B, Pre-training Token Budget=400B tokens2025.12 | 48.8 | 56.6 | 42.1 | 48.1 | 612 | |
| FineWeb-SemdedupModel Architecture=MoE, Model Parameters=7B, Pre-training Token Budget=300B tokens2025.12 | 48.3 | 57.6 | 45.3 | 50 | 112 | |
| FineWeb-DCLMModel Architecture=dense, Model Parameters=1.4B, Pre-training Token Budget=400B tokens2025.12 | 48.2 | 57.2 | 39.5 | 46.9 | 112 | |
| UltraFineWeb-enModel Architecture=MoE, Model Parameters=7B, Pre-training Token Budget=300B tokens2025.12 | 48.2 | 59.6 | 42.7 | 49.5 | 112 | |
| FineWebModel Architecture=MoE, Model Parameters=7B, Pre-training Token Budget=300B tokens2025.12 | 48.1 | 58.1 | 46.8 | 50.7 | 112 | |
| FineWebProModel Architecture=dense, Model Parameters=1.4B, Pre-training Token Budget=400B tokens2025.12 | 47.6 | 57.3 | 40.3 | 47.2 | 212 | |
| UltraFineWeb-enModel Architecture=dense, Model Parameters=1.4B, Pre-training Token Budget=400B tokens2025.12 | 47.4 | 56.5 | 38.2 | 45.8 | 12 | |
| FineWeb-EduModel Architecture=dense, Model Parameters=1.4B, Pre-training Token Budget=400B tokens2025.12 | 46.8 | 56.7 | 40.2 | 46.8 | 312 | |
| FineWeb-SemdedupModel Architecture=dense, Model Parameters=1.4B, Pre-training Token Budget=400B tokens2025.12 | 46.1 | 53.3 | 36.7 | 43.8 | 12 | |
| FineWebModel Architecture=dense, Model Parameters=1.4B, Pre-training Token Budget=400B tokens2025.12 | 46 | 53.6 | 38.8 | 44.9 | 12 |