C4
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
C4 (Final val)
18.288Perplexity
5
C4
87.2ASR
5
C4 600-token SIR-watermarked
57.7TPR @ FPR=1%
5
C4 Qwen2.5-350M (val)
16.34Perplexity
5
C4 GPT2-355M (val)
21.55PPL
5
C4 GPT2-125M (val)
29.65Perplexity (PPL)
5
C4
12.55Model Weights Share
5
C4 fr
3.34Bits per Token (B/T)
5
C4 ko
1.72C/T Ratio
5
C4 ja
1.6C/T
5
C4 zh
1.54C/T Ratio
5
C4 en
4.64Bits per Token
5
C4
7.81Latency (s)
5
C4
1F1 Score
5
C4 300K-token sample
12.24Perplexity
4
C4
5.19Perplexity (PPL)
4
C4
7.404Perplexity
4
C4 20 prompts (test)
3.2Fluency
4
C4 subset (n=512)
84.81Detection Rate (RTR 10%)
4
C4 subset (n=256)
92.35RTR (10%) Score
4
C4 subset (n=128)
97.85RTR (10%) Score
4
C4 (n=64)
99.09RTR Accuracy (10%)
4
C4 subset (n=32)
100Robustness - Random Token Replacement (10%) - Score
4
C4 subset (n=16)
100RTR (10%) Score
4
C4 RealNews
3.4Perplexity
4