Pre-training data contamination detection on PubMedQA (test)
0.54AUCProbe w. Synthetic Data
Evaluation Results
| Method | Links | |
|---|---|---|
| Probe w. Synthetic DataCategory=Our Method, Training Data=Synthetic2024.06 | 0.54 | |
| Neighbor AttackCategory=Reference-free2024.06 | 0.53 | |
| Lowercased TextCategory=Reference-based2024.06 | 0.507 | |
| Smaller ModelCategory=Reference-based, Reference Model=Contam-Small (124M Params)2024.06 | 0.495 | |
| Loss AttackCategory=Reference-free2024.06 | 0.48 | |
| Min-K% ProbCategory=Reference-free2024.06 | 0.475 | |
| Zlib CompressionCategory=Reference-based2024.06 | 0.461 |