Membership Inference on Wikipedia Pythia (train)
22.7TPR@1%FPRAttenMIA (Perturbed)
Evaluation Results
| Method | Links | |
|---|---|---|
| AttenMIA (Perturbed)Model Size=1.4b2026.01 | 22.7 | |
| AttenMIA (Perturbed)Model Size=6.9b2026.01 | 20.7 | |
| AttenMIA (Perturbed)Model Size=2.8b2026.01 | 20 | |
| AttenMIA (Transitional)Model Size=1.4b2026.01 | 16.7 | |
| RECALLModel Size=6.9b2026.01 | 13.6 | |
| RECALLModel Size=2.8b2026.01 | 11.7 | |
| RECALLModel Size=1.4b2026.01 | 10.7 | |
| AttenMIA (Transitional)Model Size=6.9b2026.01 | 7.7 | |
| AttenMIA (Transitional)Model Size=2.8b2026.01 | 7.6 | |
| BT attackModel Size=1.4b2026.01 | 6 | |
| PPL attackModel Size=1.4b2026.01 | 4.8 | |
| WS attackModel Size=6.9b2026.01 | 4.4 | |
| PPL attackModel Size=2.8b2026.01 | 4 | |
| PETALModel Size=6.9b2026.01 | 4 | |
| reference attackModel Size=1.4b2026.01 | 3.6 | |
| RS attackModel Size=6.9b2026.01 | 3.6 | |
| BT attackModel Size=6.9b2026.01 | 3.6 | |
| PPL attackModel Size=6.9b2026.01 | 3.2 | |
| RS attackModel Size=1.4b2026.01 | 3.2 | |
| PETALModel Size=1.4b2026.01 | 3.2 | |
| WS attackModel Size=1.4b2026.01 | 2.8 | |
| BT attackModel Size=2.8b2026.01 | 2.8 | |
| PETALModel Size=2.8b2026.01 | 2.8 | |
| zlib attackModel Size=2.8b2026.01 | 2.4 | |
| zlib attackModel Size=6.9b2026.01 | 2.4 | |
| zlib attackModel Size=1.4b2026.01 | 2 | |
| MIN-K% PROBModel Size=1.4b2026.01 | 2 | |
| MIN-K% PROBModel Size=6.9b2026.01 | 2 | |
| RS attackModel Size=2.8b2026.01 | 2 | |
| reference attackModel Size=6.9b2026.01 | 1.6 | |
| WS attackModel Size=2.8b2026.01 | 1.6 | |
| neighborhood attackModel Size=6.9b2026.01 | 1.2 | |
| reference attackModel Size=2.8b2026.01 | 0.8 | |
| neighborhood attackModel Size=2.8b2026.01 | 0.8 | |
| neighborhood attackModel Size=1.4b2026.01 | 0.4 | |
| MIN-K% PROBModel Size=2.8b2026.01 | 0.4 |