Membership Inference on GitHub Pythia (train)
95.6TPR@1%FPRAttenMIA (Perturbed)
Evaluation Results
| Method | Links | |
|---|---|---|
| AttenMIA (Perturbed)Model Size=2.8b2026.01 | 95.6 | |
| AttenMIA (Perturbed)Model Size=6.9b2026.01 | 95.4 | |
| AttenMIA (Perturbed)Model Size=1.4b2026.01 | 91.3 | |
| AttenMIA (Transitional)Model Size=2.8b2026.01 | 73.8 | |
| AttenMIA (Transitional)Model Size=1.4b2026.01 | 60.7 | |
| AttenMIA (Transitional)Model Size=6.9b2026.01 | 60.1 | |
| RECALLModel Size=6.9b2026.01 | 52.7 | |
| RECALLModel Size=2.8b2026.01 | 49.6 | |
| PPL attackModel Size=6.9b2026.01 | 48.8 | |
| RECALLModel Size=1.4b2026.01 | 48.8 | |
| PETALModel Size=6.9b2026.01 | 44.4 | |
| PPL attackModel Size=2.8b2026.01 | 43.2 | |
| zlib attackModel Size=6.9b2026.01 | 36.8 | |
| PPL attackModel Size=1.4b2026.01 | 35.6 | |
| RS attackModel Size=6.9b2026.01 | 35.6 | |
| PETALModel Size=2.8b2026.01 | 33.6 | |
| neighborhood attackModel Size=1.4b2026.01 | 32 | |
| RS attackModel Size=2.8b2026.01 | 32 | |
| MIN-K% PROBModel Size=2.8b2026.01 | 31.6 | |
| zlib attackModel Size=2.8b2026.01 | 31.2 | |
| PETALModel Size=1.4b2026.01 | 31.2 | |
| RS attackModel Size=1.4b2026.01 | 28.8 | |
| MIN-K% PROBModel Size=6.9b2026.01 | 28.4 | |
| zlib attackModel Size=1.4b2026.01 | 27.6 | |
| neighborhood attackModel Size=6.9b2026.01 | 27.2 | |
| neighborhood attackModel Size=2.8b2026.01 | 26 | |
| BT attackModel Size=2.8b2026.01 | 25.2 | |
| BT attackModel Size=6.9b2026.01 | 25.2 | |
| MIN-K% PROBModel Size=1.4b2026.01 | 23.6 | |
| BT attackModel Size=1.4b2026.01 | 22.8 | |
| WS attackModel Size=6.9b2026.01 | 21.6 | |
| WS attackModel Size=2.8b2026.01 | 17.2 | |
| reference attackModel Size=1.4b2026.01 | 15.2 | |
| WS attackModel Size=1.4b2026.01 | 13.6 | |
| reference attackModel Size=2.8b2026.01 | 12.8 | |
| reference attackModel Size=6.9b2026.01 | 9.6 |