Membership Inference Attack on DM Math Pythia
100ROC AUCAttenMIA (Perturbed)
Evaluation results
| Method | Links | |
|---|---|---|
| AttenMIA (Perturbed)Model size=1.4b2026.01 | 100 | |
| AttenMIA (Perturbed)Model size=2.8b2026.01 | 100 | |
| AttenMIA (Perturbed)Model size=6.9b2026.01 | 100 | |
| AttenMIA (Transitional)Model size=6.9b2026.01 | 96 | |
| AttenMIA (Transitional)Model size=2.8b2026.01 | 95 | |
| AttenMIA (Transitional)Model size=1.4b2026.01 | 94 | |
| PPL attackModel size=1.4b2026.01 | 88 | |
| PPL attackModel size=2.8b2026.01 | 87 | |
| PETALModel size=1.4b2026.01 | 87 | |
| PPL attackModel size=6.9b2026.01 | 86 | |
| PETALModel size=2.8b2026.01 | 85 | |
| PETALModel size=6.9b2026.01 | 85 | |
| reference attackModel size=1.4b2026.01 | 82 | |
| zlib attackModel size=1.4b2026.01 | 80 | |
| reference attackModel size=2.8b2026.01 | 78 | |
| zlib attackModel size=2.8b2026.01 | 78 | |
| reference attackModel size=6.9b2026.01 | 77 | |
| zlib attackModel size=6.9b2026.01 | 77 | |
| neighborhood attackModel size=1.4b2026.01 | 73 | |
| neighborhood attackModel size=6.9b2026.01 | 73 | |
| neighborhood attackModel size=2.8b2026.01 | 72 | |
| MIN-K% PROBModel size=1.4b2026.01 | 71 | |
| MIN-K% PROBModel size=6.9b2026.01 | 68 | |
| MIN-K% PROBModel size=2.8b2026.01 | 67 | |
| RECALLModel size=1.4b2026.01 | 53 | |
| RECALLModel size=2.8b2026.01 | 51 | |
| RECALLModel size=6.9b2026.01 | 51 | |
| WS attackModel size=2.8b2026.01 | 49 | |
| WS attackModel size=1.4b2026.01 | 45 | |
| RS attackModel size=2.8b2026.01 | 45 | |
| BT attackModel size=2.8b2026.01 | 45 | |
| WS attackModel size=6.9b2026.01 | 44 | |
| RS attackModel size=6.9b2026.01 | 43 | |
| RS attackModel size=1.4b2026.01 | 42 | |
| BT attackModel size=1.4b2026.01 | 39 | |
| BT attackModel size=6.9b2026.01 | 39 |