Membership Inference Attack on GitHub Pythia
1ROC AUCAttenMIA (Perturbed)
Evaluation Results
| Method | Links | |
|---|---|---|
| AttenMIA (Perturbed)Model size=6.9b2026.01 | 1 | |
| AttenMIA (Perturbed)Model size=1.4b2026.01 | 0.99 | |
| AttenMIA (Perturbed)Model size=2.8b2026.01 | 0.99 | |
| AttenMIA (Transitional)Model size=2.8b2026.01 | 0.93 | |
| AttenMIA (Transitional)Model size=6.9b2026.01 | 0.89 | |
| PPL attackModel size=6.9b2026.01 | 0.88 | |
| MIN-K% PROBModel size=6.9b2026.01 | 0.88 | |
| AttenMIA (Transitional)Model size=1.4b2026.01 | 0.88 | |
| PPL attackModel size=2.8b2026.01 | 0.87 | |
| zlib attackModel size=6.9b2026.01 | 0.87 | |
| PETALModel size=6.9b2026.01 | 0.87 | |
| PPL attackModel size=1.4b2026.01 | 0.86 | |
| zlib attackModel size=2.8b2026.01 | 0.86 | |
| MIN-K% PROBModel size=1.4b2026.01 | 0.86 | |
| MIN-K% PROBModel size=2.8b2026.01 | 0.86 | |
| zlib attackModel size=1.4b2026.01 | 0.85 | |
| PETALModel size=1.4b2026.01 | 0.85 | |
| PETALModel size=2.8b2026.01 | 0.85 | |
| neighborhood attackModel size=6.9b2026.01 | 0.84 | |
| neighborhood attackModel size=1.4b2026.01 | 0.83 | |
| neighborhood attackModel size=2.8b2026.01 | 0.83 | |
| RS attackModel size=6.9b2026.01 | 0.78 | |
| WS attackModel size=6.9b2026.01 | 0.77 | |
| RS attackModel size=1.4b2026.01 | 0.77 | |
| RS attackModel size=2.8b2026.01 | 0.77 | |
| BT attackModel size=6.9b2026.01 | 0.77 | |
| WS attackModel size=1.4b2026.01 | 0.76 | |
| WS attackModel size=2.8b2026.01 | 0.76 | |
| BT attackModel size=2.8b2026.01 | 0.76 | |
| BT attackModel size=1.4b2026.01 | 0.74 | |
| RECALLModel size=6.9b2026.01 | 0.74 | |
| RECALLModel size=2.8b2026.01 | 0.72 | |
| RECALLModel size=1.4b2026.01 | 0.7 | |
| reference attackModel size=1.4b2026.01 | 0.65 | |
| reference attackModel size=2.8b2026.01 | 0.65 | |
| reference attackModel size=6.9b2026.01 | 0.64 |