Clustered Pre-training Data
Benchmarks
TaskDatasetSOTA resultTrendResults
Clustered Pre-training Data Average
18.008Perplexity (PPL)
2
Clustered Pre-training Data Cluster 15
18.294Perplexity (PPL)
2
Clustered Pre-training Data Cluster 14
20.656Perplexity (PPL)
2
Clustered Pre-training Data Cluster 13
19.461Perplexity (PPL)
2
Clustered Pre-training Data Cluster 12
17.731Perplexity (PPL)
2
Clustered Pre-training Data Cluster 11
22.002Perplexity (PPL)
2
Clustered Pre-training Data Cluster 10
17.783Perplexity (PPL)
2
Clustered Pre-training Data Cluster 9
15.405Perplexity (PPL)
2
Clustered Pre-training Data Cluster 8
14.96Perplexity (PPL)
2
Clustered Pre-training Data Cluster 7
15.972Perplexity (PPL)
2
Clustered Pre-training Data Cluster 6
15.097Perplexity (PPL)
2
Clustered Pre-training Data Cluster 5
19.55Perplexity (PPL)
2
Clustered Pre-training Data Cluster 4
18.414Perplexity (PPL)
2
Clustered Pre-training Data Cluster 3
16.903Perplexity (PPL)
2
Clustered Pre-training Data Cluster 2
18.354Perplexity (PPL)
2
Clustered Pre-training Data Cluster 1
17.731Perplexity (PPL)
2
Clustered Pre-training Data Cluster 0
19.819Perplexity (PPL)
2