Membership Inference Attack on SQuADShifts (test)
0.883AUCreference
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| referencetarget model=Pythia-1.4B, k=1, p=0.22026.05 | 0.883 | 17.8 | 45.4 | 71.1 | |
| referencetarget model=Gemma-1.1-2B-IT, k=1, p=12026.05 | 0.882 | 8.5 | 47 | 74.8 | |
| referencetarget model=Gemma-1.1-2B-IT, k=1, p=0.22026.05 | 0.876 | 11.9 | 39 | 66.2 | |
| referencetarget model=Pythia-1.4B, k=1, p=12026.05 | 0.876 | 18.2 | 45.4 | 73.8 | |
| referencetarget model=Pythia-1.4B, k=1, p=0.12026.05 | 0.856 | 6.7 | 38.7 | 63.2 | |
| referencetarget model=Gemma-1.1-2B-IT, k=1, p=0.12026.05 | 0.854 | 12.2 | 37 | 64.6 | |
| predictions onlytarget model=Gemma-1.1-2B-IT, k=1, p=12026.05 | 0.785 | 15.5 | 31.8 | 48.4 | |
| predictions onlytarget model=Gemma-1.1-2B-IT, k=1, p=0.82026.05 | 0.781 | 15.3 | 31.6 | 47.4 | |
| predictions onlytarget model=Gemma-1.1-2B-IT, k=1, p=0.52026.05 | 0.768 | 14.6 | 30.1 | 44.1 | |
| predictions onlytarget model=Gemma-1.1-2B-IT, k=1, p=0.32026.05 | 0.746 | 13.3 | 29.4 | 40.5 | |
| brainwashtarget model=Gemma-1.1-2B-IT, k=12026.05 | 0.736 | 11.2 | 31 | 40.5 | |
| predictions onlytarget model=Gemma-1.1-2B-IT, k=1, p=0.22026.05 | 0.72 | 12.9 | 27.3 | 37.7 | |
| predictions onlytarget model=Pythia-1.4B, k=1, p=12026.05 | 0.711 | 10.7 | 24.9 | 35.2 | |
| predictions onlytarget model=Pythia-1.4B, k=1, p=0.82026.05 | 0.706 | 10.6 | 23.8 | 33.6 | |
| predictions onlytarget model=Pythia-1.4B, k=1, p=0.52026.05 | 0.692 | 10.5 | 22.1 | 31.2 | |
| predictions onlytarget model=Pythia-1.4B, k=1, p=0.32026.05 | 0.67 | 9.2 | 20 | 29.7 | |
| predictions onlytarget model=Gemma-1.1-2B-IT, k=1, p=0.12026.05 | 0.66 | 11.7 | 22.1 | 31.3 | |
| predictions onlytarget model=Pythia-1.4B, k=1, p=0.22026.05 | 0.648 | 8.6 | 17.9 | 28.1 | |
| repeattarget model=Pythia-1.4B, k=12026.05 | 0.622 | 17.4 | 23.6 | 30.8 | |
| logittarget model=Pythia-1.4B, k=12026.05 | 0.619 | 11.4 | 17.1 | 22.4 | |
| logittarget model=Gemma-1.1-2B-IT, k=12026.05 | 0.608 | 6 | 12.3 | 18.4 | |
| predictions onlytarget model=Pythia-1.4B, k=1, p=0.12026.05 | 0.605 | 7.2 | 14.7 | 23.8 | |
| predictions onlytarget model=Gemma-1.1-2B-IT, k=1, p=0.052026.05 | 0.597 | 5.8 | 16.8 | 25.2 | |
| predictions onlytarget model=Pythia-1.4B, k=1, p=0.052026.05 | 0.568 | 2.2 | 8.7 | 16.1 | |
| repeattarget model=Gemma-1.1-2B-IT, k=12026.05 | 0.564 | 9 | 14.9 | 22.8 | |
| brainwashtarget model=Pythia-1.4B, k=12026.05 | 0.502 | 1 | 5 | 10 |