Membership Inference Attack on WikiMIA (TPR@5%FPR)
19.6TPR@5%FPRProbe Attack (Probe w. Synthetic Data)
Evaluation results
| Method | Links | |
|---|---|---|
| Probe Attack (Probe w. Synthetic Data)Target Model=Pythia2024.06 | 19.6 | |
| Zlib CompressionTarget Model=Pythia2024.06 | 17.3 | |
| Min-K% ProbTarget Model=Pythia2024.06 | 16.9 | |
| Probe Attack (Probe w. Real Data)Target Model=Pythia2024.06 | 16.7 | |
| Probe Attack (Probe w. Real Data)Target Model=OPT2024.06 | 15.4 | |
| Min-K% ProbTarget Model=OPT2024.06 | 15 | |
| Zlib CompressionTarget Model=OPT2024.06 | 14.4 | |
| Neighbor AttackTarget Model=Pythia2024.06 | 14 | |
| Smaller ModelTarget Model=Pythia2024.06 | 14 | |
| Loss AttackTarget Model=Pythia2024.06 | 13.7 | |
| Neighbor AttackTarget Model=OPT2024.06 | 13.4 | |
| Loss AttackTarget Model=OPT2024.06 | 11.4 | |
| Smaller ModelTarget Model=OPT2024.06 | 10.5 | |
| Probe Attack (Probe w. Synthetic Data)Target Model=OPT2024.06 | 10.5 | |
| Lowercased TextTarget Model=Pythia2024.06 | 10.1 | |
| Lowercased TextTarget Model=OPT2024.06 | 9.1 |