Language Modeling on Multilingual Held-out Corpus Normalized Sum
8.199CE LossShapleyLaw
Evaluation results
| Method | Links | |
|---|---|---|
| ShapleyLawLanguage Mixture Ratio (en)=0.1, Language Mixture Ratio (de)=18.7, Language Mixture Ratio (fr)=15.0, Language Mixture Ratio (es)=11.5, Language Mixture Ratio (zh)=20.8, Language Mixture Ratio (ja)=15.3, Language Mixture Ratio (ko)=18.62026.03 | 8.199 | |
| α = 0.5Language Mixture Ratio (en)=14.8, Language Mixture Ratio (de)=14.8, Language Mixture Ratio (fr)=16.2, Language Mixture Ratio (es)=14.1, Language Mixture Ratio (zh)=15.2, Language Mixture Ratio (ja)=21.4, Language Mixture Ratio (ko)=12.82026.03 | 8.258 | |
| UniformLanguage Mixture Ratio (en)=14.3, Language Mixture Ratio (de)=14.3, Language Mixture Ratio (fr)=14.3, Language Mixture Ratio (es)=14.3, Language Mixture Ratio (zh)=14.3, Language Mixture Ratio (ja)=14.3, Language Mixture Ratio (ko)=14.22026.03 | 8.269 | |
| Common CrawlLanguage Mixture Ratio (en)=44.4, Language Mixture Ratio (de)=12.1, Language Mixture Ratio (fr)=9.8, Language Mixture Ratio (es)=9.5, Language Mixture Ratio (zh)=12.0, Language Mixture Ratio (ja)=10.7, Language Mixture Ratio (ko)=1.52026.03 | 8.285 | |
| FamilyLawLanguage Mixture Ratio (en)=8.3, Language Mixture Ratio (de)=8.3, Language Mixture Ratio (fr)=8.3, Language Mixture Ratio (es)=8.2, Language Mixture Ratio (zh)=26.3, Language Mixture Ratio (ja)=19.5, Language Mixture Ratio (ko)=21.12026.03 | 8.327 | |
| FineWeb-2Language Mixture Ratio (en)=81.3, Language Mixture Ratio (de)=3.3, Language Mixture Ratio (fr)=2.3, Language Mixture Ratio (es)=2.7, Language Mixture Ratio (zh)=6.8, Language Mixture Ratio (ja)=3.1, Language Mixture Ratio (ko)=0.52026.03 | 8.616 |