Membership Inference Attack on SQuADShifts
0.9AUCreference
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| referenceTarget Model=Gemma-7B-IT, k=1, p=12026.05 | 0.9 | 19.7 | 58.5 | 75.8 | |
| referenceTarget Model=Gemma-7B-IT, k=1, p=0.12026.05 | 0.894 | 17.4 | 51.5 | 75 | |
| referenceTarget Model=Gemma-7B-IT, k=1, p=0.22026.05 | 0.893 | 14.8 | 48.6 | 75 | |
| logitTarget Model=Gemma-7B-IT, k=1, p=-2026.05 | 0.84 | 4.6 | 22.8 | 44.9 | |
| prediction onlyTarget Model=Gemma-7B-IT, k=1, p=0.052026.05 | 0.636 | 4.1 | 12.5 | 20.6 | |
| prediction onlyTarget Model=Gemma-7B-IT, k=1, p=0.22026.05 | 0.627 | 2.6 | 12.7 | 23.3 | |
| prediction onlyTarget Model=Gemma-7B-IT, k=1, p=0.12026.05 | 0.623 | 3.7 | 9.8 | 18.3 | |
| prediction onlyTarget Model=Gemma-7B-IT, k=1, p=0.32026.05 | 0.606 | 2.4 | 11.2 | 17.2 | |
| prediction onlyTarget Model=Gemma-7B-IT, k=1, p=0.82026.05 | 0.593 | 2.4 | 9.3 | 20.5 | |
| prediction onlyTarget Model=Gemma-7B-IT, k=1, p=0.52026.05 | 0.581 | 2.5 | 13.2 | 22.2 | |
| prediction onlyTarget Model=Gemma-7B-IT, k=1, p=12026.05 | 0.573 | 2.3 | 7.9 | 15.5 | |
| repeatTarget Model=Gemma-7B-IT, k=1, p=-2026.05 | 0.541 | 2 | 7.6 | 14.1 | |
| brainwashTarget Model=Gemma-7B-IT, k=1, p=-2026.05 | 0.529 | 1.1 | 5.6 | 11.2 |