Contamination Detection on WikiMIA
74.3AUCProbe w. Real Data
Evaluation results
| Method | Links | |
|---|---|---|
| Probe w. Real DataTarget Model=TinyLLaMA, Category=Our Method2024.06 | 74.3 | |
| Probe w. Real DataTarget Model=Pythia2024.06 | 69.8 | |
| Probe w. Synthetic DataTarget Model=Pythia2024.06 | 69.4 | |
| Probe w. Real DataTarget Model=OPT2024.06 | 68.1 | |
| Probe w. Synthetic DataTarget Model=OPT2024.06 | 66.2 | |
| Smaller ModelTarget Model=OPT2024.06 | 65.8 | |
| Smaller ModelTarget Model=Pythia2024.06 | 65.5 | |
| Lowercased TextTarget Model=Pythia2024.06 | 64.7 | |
| Loss AttackTarget Model=Pythia2024.06 | 63.9 | |
| Zlib CompressionTarget Model=Pythia2024.06 | 63.8 | |
| Min-K% ProbTarget Model=OPT2024.06 | 63.2 | |
| Loss AttackTarget Model=OPT2024.06 | 63 | |
| Zlib CompressionTarget Model=OPT2024.06 | 62.9 | |
| Probe w. Real Data (LoRA)Target Model=TinyLLaMA, Category=Our Method, Adaptation=LoRA2024.06 | 62.7 | |
| Min-K% ProbTarget Model=Pythia2024.06 | 62.7 | |
| Neighbor AttackTarget Model=Pythia2024.06 | 62.1 | |
| Lowercased TextTarget Model=OPT2024.06 | 61.6 | |
| Neighbor AttackTarget Model=OPT2024.06 | 58.5 | |
| Zlib CompressionTarget Model=TinyLLaMA, Category=Reference-based Methods2024.06 | 56.7 | |
| Loss AttackTarget Model=TinyLLaMA, Category=Reference-free Methods2024.06 | 55.7 | |
| Min-K% ProbTarget Model=TinyLLaMA, Category=Reference-free Methods2024.06 | 52.9 | |
| Lowercased TextTarget Model=TinyLLaMA, Category=Reference-based Methods2024.06 | 49.8 | |
| Neighbor AttackTarget Model=TinyLLaMA, Category=Reference-free Methods2024.06 | 48.8 |