Pre-training Data Detection on ArxivMIA Math
57.4AUCProbe w. Synthetic Data
Evaluation results
| Method | Links | |
|---|---|---|
| Probe w. Synthetic DataTarget Model=OpenLLaMA2024.06 | 57.4 | |
| Probe w. Real DataTarget Model=OpenLLaMA2024.06 | 56.9 | |
| Smaller ModelTarget Model=OpenLLaMA2024.06 | 56.7 | |
| Probe w. Synthetic DataTarget Model=TinyLLaMA2024.06 | 56.7 | |
| Probe w. Real DataTarget Model=TinyLLaMA2024.06 | 56.1 | |
| Neighbor AttackTarget Model=OpenLLaMA2024.06 | 54.1 | |
| Neighbor AttackTarget Model=TinyLLaMA2024.06 | 53.4 | |
| Lowercased TextTarget Model=OpenLLaMA2024.06 | 50.8 | |
| Min-K% ProbTarget Model=OpenLLaMA2024.06 | 48.5 | |
| Lowercased TextTarget Model=TinyLLaMA2024.06 | 48.4 | |
| Loss AttackTarget Model=OpenLLaMA2024.06 | 47.4 | |
| Min-K% ProbTarget Model=TinyLLaMA2024.06 | 45.8 | |
| Loss AttackTarget Model=TinyLLaMA2024.06 | 44.9 | |
| Zlib CompressionTarget Model=OpenLLaMA2024.06 | 44.7 | |
| Zlib CompressionTarget Model=TinyLLaMA2024.06 | 44 |