C4
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
C4 LLaMA3-8B-BASE Max 128 Tokens
1AUC
20
C4 T5 (val)
15.82PPLX
20
C4 en (train)
2.94Train Loss
18
C4 Message length b=8 v1 (test)
100BA
18
C4
6.811Perplexity
18
C4 Gemma-7B
1ROC-AUC
18
C4 Mistral-7B (val)
100TPR
18
C4 Llama-7B (val)
100TPR
18
C4 Watermarked
6.375PPL
17
C4 Post-translation tr-fr
49.8TPR @ 1% FPR
17
C4 Post-translation tr-de
65TPR @ 1% FPR
17
C4 Post-paraphrasing
74.6TPR @ 1% FPR
17
C4 Unmodified
100TPR @ 1% FPR
17
C4
15.57Perplexity
16
C4
12.701Perplexity
16
C4 RealNewsLike
100AUC
16
C4 Llama-1B scratch (val)
2.638Validation Loss
16
C4 (val)
19.2OLMo Perplexity
15
C4
99.88TPR @ FPR=1%
15
C4-RealNewsLike Translate (test)
97.3AUC
14
C4 RealNewsLike Dipper-2 (test)
85.9AUC
14
C4-RealNewsLike Dipper-1 (test)
0.977AUC
14
C4 RealNewsLike (Sub-0.5) (test)
86.2AUC
14
C4 RealNewsLike (Del-0.5) (test)
85.1AUC
14
C4 (val)
13.33PPL (60k)
14