The Pile
Benchmarks
TaskDatasetSOTA resultTrendResults
The Pile
2.53Perplexity
132
The Pile (val)
0.62Perplexity (bits/byte)
55
The Pile (test)
9.213PPL (The Pile Test)
53
The Pile (val)
4.24PPL
41
The Pile deduplicated (val)
7.14Perplexity
22
The Pile non-copyrighted (test)
0.557BPB
20
The Pile 32 sample (val)
0EL10 (%)
15
The Pile
0.927AUROC
14
The Pile (eval)
14.1Perplexity (PPL)
12
The Pile
4.05Inference Speedup
11
The Pile (train)
45Exact Extract Rate
10
The Pile (test)
63.4Fractional Extraction Rate
10
The Pile non-copyrighted without Wikipedia (test)
0.559BPB
8
The Pile Downstream Evaluation Suite
29.7HellaSwag Accuracy
7
The PILE (train test)
66.5Loss
7
The Pile (train)
5,144Wall Time (PPL=42)
6
The Pile Wikipedia (test)
66.71BPB
6
The Pile USPTO (test)
65.7BPB
6
The Pile StackEx. (test)
80.37BPB (%)
6
The Pile PubMed Cent. (test)
84.13BPB%
6
The Pile PubMed Abs. (test)
87.7BPB (%)
6
The Pile NIH (test)
66.05BPB
6
The Pile HackerNews (test)
0.858BPB (Bits Per Byte)
6
The Pile Enron (test)
57.32BPB
6
The Pile Github (test)
40.88Bits Per Byte (BPB)
6