Language Modeling on FineWeb-10B Nemotron-10B sample (val)
14.78Validation PerplexityPRISM
Evaluation results
| Method | Links | |
|---|---|---|
| PRISMData=FineWeb, Training Tokens=20B, Model Parameters=1.6B2026.07 | 14.78 | |
| GPTData=FineWeb, Training Tokens=10B, Model Parameters=1.6B2026.07 | 15.24 | |
| PRISMData=Nemotron, Training Tokens=50B, Model Parameters=1.6B2026.07 | 15.26 | |
| GPTData=Nemotron, Training Tokens=10B, Model Parameters=1.6B2026.07 | 15.5 | |
| PRISMData=Nemotron, Training Tokens=40B, Model Parameters=1.6B2026.07 | 15.59 | |
| PRISMData=FineWeb, Training Tokens=10B, Model Parameters=1.6B2026.07 | 15.96 | |
| GPTData=FineWeb, Training Tokens=10B, Model Parameters=760M2026.07 | 16.18 | |
| PRISMData=Nemotron, Training Tokens=30B, Model Parameters=1.6B2026.07 | 16.62 | |
| PRISMData=Nemotron, Training Tokens=20B, Model Parameters=1.6B2026.07 | 17.19 | |
| GPTData=FineWeb, Training Tokens=10B, Model Parameters=350M2026.07 | 17.83 | |
| PRISMData=Nemotron, Training Tokens=10B, Model Parameters=1.6B2026.07 | 17.9 | |
| PRISMData=FineWeb, Training Tokens=10B, Model Parameters=760M2026.07 | 18.38 | |
| PRISMData=FineWeb, Training Tokens=10B, Model Parameters=350M2026.07 | 20.22 | |
| GPTData=FineWeb, Training Tokens=10B, Model Parameters=124M2026.07 | 21.45 | |
| PRISMData=FineWeb, Training Tokens=10B, Model Parameters=124M2026.07 | 23.56 |