C4
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
C4
1.787Log Perplexity
4
C4 (first 100 prompts)
100EM
3
C4
4.86Evaluation Loss (10k Steps)
3
C4 TinyLlama-1.1B-Chat v1.0
9.75Perplexity (C4)
3
C4 100 GB (sampled)
58.9Generation Time (s)
3
C4 Paraphrase Dipper
76.4Bit Accuracy
3
C4 Deletion, epsilon = 0.1
88.11Bit Accuracy
3
C4 Synonym, epsilon = 0.2
97.12Bit Accuracy
3
C4 Copy-paste, epsilon = 0.2
97.3Bit Accuracy
3
C4 No attack
98.93Bit Accuracy
3
C4
11.96Perplexity
2
Full C4 (365M documents)
27,207,144Number of Duplicates
2
C4 Sampled 50M documents
2,766,436Number of Duplicates
2
C4 Mistral-7B
62KGW-WM IoU
2
C4 Llama-7B
65.7KGW-WM IoU
2
C4
0.255L2 Distance
1
C4 Low and Slow
10Initial Peak Psi
1
C4 (test)
—Primary metric
0