Membership Inference Attack on OLMo-7B
55.3TPR@1%FPRCAMIA + Combined
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CAMIA + CombinedBase Attack=CAMIA, Augmentation=Combined, Iterations=152026.06 | 55.3 | 96.9 | |
| CAMIA + DiversityBase Attack=CAMIA, Augmentation=Diversity, Iterations=152026.06 | 51.7 | 96.6 | |
| CAMIA + QualityBase Attack=CAMIA, Augmentation=Quality, Iterations=152026.06 | 50.1 | 96.4 | |
| CAMIABase Attack=CAMIA, Augmentation=None, Iterations=152026.06 | 42.8 | 95.8 | |
| Zlib + DiversityBase Attack=Zlib, Augmentation=Diversity, Iterations=152026.06 | 31.8 | 84.2 | |
| Loss + DiversityBase Attack=Loss, Augmentation=Diversity, Iterations=152026.06 | 30.3 | 73.5 | |
| Zlib + CombinedBase Attack=Zlib, Augmentation=Combined, Iterations=152026.06 | 29.5 | 86.8 | |
| Loss + CombinedBase Attack=Loss, Augmentation=Combined, Iterations=152026.06 | 26.3 | 80.4 | |
| Min-K% + CombinedBase Attack=Min-K%, Augmentation=Combined, Iterations=152026.06 | 24 | 83.7 | |
| Min-K% + DiversityBase Attack=Min-K%, Augmentation=Diversity, Iterations=152026.06 | 21.9 | 77.5 | |
| Zlib + QualityBase Attack=Zlib, Augmentation=Quality, Iterations=152026.06 | 20.8 | 83.3 | |
| Min-K% + QualityBase Attack=Min-K%, Augmentation=Quality, Iterations=152026.06 | 9.5 | 77.2 | |
| Min-K%Base Attack=Min-K%, Augmentation=None, Iterations=152026.06 | 6.7 | 70.3 | |
| Loss + QualityBase Attack=Loss, Augmentation=Quality, Iterations=152026.06 | 3.2 | 70.2 | |
| ZlibBase Attack=Zlib, Augmentation=None, Iterations=152026.06 | 2.2 | 17.9 | |
| LossBase Attack=Loss, Augmentation=None, Iterations=152026.06 | 0.8 | 52.3 |