Membership Inference Attack on C4 Vocabulary Size: 110,000
71.8AUCVocabulary Overlap
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Vocabulary OverlapShadow Tokenizers=true, Auxiliary Datasets=true2025.10 | 71.8 | 67.2 | 28.75 | |
| Frequency EstimationShadow Tokenizers=true, Auxiliary Datasets=true2025.10 | 64.5 | 64.1 | 22 | |
| Naive BayesShadow Tokenizers=false, Auxiliary Datasets=true, k=60,0002025.10 | 55.1 | 53.8 | 5.57 | |
| Naive BayesShadow Tokenizers=false, Auxiliary Datasets=true, k=40,0002025.10 | 54.6 | 53.3 | 6.1 | |
| Naive BayesShadow Tokenizers=false, Auxiliary Datasets=true, k=20,0002025.10 | 52.6 | 52.4 | 3.44 | |
| Compression RateShadow Tokenizers=false, Auxiliary Datasets=false2025.10 | 50.9 | 51.7 | 0.71 | |
| Merge SimilarityShadow Tokenizers=true, Auxiliary Datasets=true2025.10 | 49.4 | 50.7 | 1.02 |