Membership Inference Attack on HackerNews Pythia
0.72ROC AUCAttenMIA (Perturbed)
Evaluation Results
| Method | Links | |
|---|---|---|
| AttenMIA (Perturbed)Model size=2.8b2026.01 | 0.72 | |
| AttenMIA (Perturbed)Model size=6.9b2026.01 | 0.72 | |
| AttenMIA (Perturbed)Model size=1.4b2026.01 | 0.7 | |
| AttenMIA (Transitional)Model size=1.4b2026.01 | 0.62 | |
| PPL attackModel size=2.8b2026.01 | 0.6 | |
| PPL attackModel size=6.9b2026.01 | 0.6 | |
| zlib attackModel size=2.8b2026.01 | 0.6 | |
| PPL attackModel size=1.4b2026.01 | 0.59 | |
| zlib attackModel size=1.4b2026.01 | 0.59 | |
| zlib attackModel size=6.9b2026.01 | 0.59 | |
| MIN-K% PROBModel size=6.9b2026.01 | 0.59 | |
| PETALModel size=1.4b2026.01 | 0.59 | |
| AttenMIA (Transitional)Model size=2.8b2026.01 | 0.59 | |
| AttenMIA (Transitional)Model size=6.9b2026.01 | 0.59 | |
| MIN-K% PROBModel size=2.8b2026.01 | 0.58 | |
| PETALModel size=2.8b2026.01 | 0.58 | |
| PETALModel size=6.9b2026.01 | 0.58 | |
| MIN-K% PROBModel size=1.4b2026.01 | 0.57 | |
| neighborhood attackModel size=2.8b2026.01 | 0.55 | |
| neighborhood attackModel size=6.9b2026.01 | 0.54 | |
| RECALLModel size=6.9b2026.01 | 0.54 | |
| reference attackModel size=2.8b2026.01 | 0.53 | |
| neighborhood attackModel size=1.4b2026.01 | 0.53 | |
| BT attackModel size=6.9b2026.01 | 0.53 | |
| RECALLModel size=1.4b2026.01 | 0.53 | |
| RECALLModel size=2.8b2026.01 | 0.53 | |
| reference attackModel size=1.4b2026.01 | 0.52 | |
| reference attackModel size=6.9b2026.01 | 0.52 | |
| BT attackModel size=2.8b2026.01 | 0.52 | |
| WS attackModel size=1.4b2026.01 | 0.51 | |
| WS attackModel size=2.8b2026.01 | 0.51 | |
| RS attackModel size=2.8b2026.01 | 0.51 | |
| RS attackModel size=6.9b2026.01 | 0.51 | |
| BT attackModel size=1.4b2026.01 | 0.51 | |
| RS attackModel size=1.4b2026.01 | 0.49 | |
| WS attackModel size=6.9b2026.01 | 0.48 |