C4
Benchmarks
TaskDatasetSOTA resultTrendResults
C4
1Perplexity
1,688
C4
4.77Perplexity
1,565
C4 (val)
1.7PPL
908
C4 (test)
4.97Perplexity
488
C4
5.52Perplexity
482
C4
5.52Perplexity
278
C4
6.24Perplexity
137
C4
2.55C4 Loss
121
C4
1TPR @ FPR=1%
95
C4
4.21PPL
79
C4
9.36Perplexity
58
C4 (val)
17.8Perplexity
58
C4 (train)
15.28PPL
50
C4
13.3Perplexity
47
C4 Llama 2 pre-training (val)
13.19Perplexity
47
C4
100AUROC
40
C4
100TPR (FPR < 10^-4)
40
C4 LLaMA-130M (val)
18.504Perplexity
40
C4
1Entropy
39
C4
2.7-log(DKL)
36
C4
100TPR @ 1% FPR
36
C4
2.834Log-PPL
35
C4 (val)
3.828PPLX
35
C4
100Cosine Similarity
32
C4
18.08Word Perplexity
32