Language Modeling on Multilingual Held-out Corpus Unweighted Sum
18.289CE LossShapleyLaw
Evaluation results
| Method | Links | |
|---|---|---|
| ShapleyLawLanguage Mixture Ratio (en)=1.8, Language Mixture Ratio (de)=16.9, Language Mixture Ratio (fr)=12.6, Language Mixture Ratio (es)=9.6, Language Mixture Ratio (zh)=20.1, Language Mixture Ratio (ja)=19.8, Language Mixture Ratio (ko)=19.22026.03 | 18.289 | |
| α = 0.5Language Mixture Ratio (en)=14.8, Language Mixture Ratio (de)=14.8, Language Mixture Ratio (fr)=16.2, Language Mixture Ratio (es)=14.1, Language Mixture Ratio (zh)=15.2, Language Mixture Ratio (ja)=21.4, Language Mixture Ratio (ko)=12.82026.03 | 18.349 | |
| UniformLanguage Mixture Ratio (en)=14.3, Language Mixture Ratio (de)=14.3, Language Mixture Ratio (fr)=14.3, Language Mixture Ratio (es)=14.3, Language Mixture Ratio (zh)=14.3, Language Mixture Ratio (ja)=14.3, Language Mixture Ratio (ko)=14.22026.03 | 18.354 | |
| FamilyLawLanguage Mixture Ratio (en)=8.1, Language Mixture Ratio (de)=8.1, Language Mixture Ratio (fr)=8.1, Language Mixture Ratio (es)=8.1, Language Mixture Ratio (zh)=24.2, Language Mixture Ratio (ja)=23.1, Language Mixture Ratio (ko)=20.32026.03 | 18.435 | |
| Common CrawlLanguage Mixture Ratio (en)=44.4, Language Mixture Ratio (de)=12.1, Language Mixture Ratio (fr)=9.8, Language Mixture Ratio (es)=9.5, Language Mixture Ratio (zh)=12.0, Language Mixture Ratio (ja)=10.7, Language Mixture Ratio (ko)=1.52026.03 | 18.679 | |
| FineWeb-2Language Mixture Ratio (en)=81.3, Language Mixture Ratio (de)=3.3, Language Mixture Ratio (fr)=2.3, Language Mixture Ratio (es)=2.7, Language Mixture Ratio (zh)=6.8, Language Mixture Ratio (ja)=3.1, Language Mixture Ratio (ko)=0.52026.03 | 19.357 |