List your company
ResearchFormerly WizWand

Membership Inference Attack on GitHub Pythia

1ROC AUCAttenMIA (Perturbed)
0.62560.72280.820.9172Jan 26, 2026
Updated 8mo ago

Evaluation results

MethodLinks
AttenMIA (Perturbed)Model size=6.9b2026.011
AttenMIA (Perturbed)Model size=1.4b2026.010.99
AttenMIA (Perturbed)Model size=2.8b2026.010.99
AttenMIA (Transitional)Model size=2.8b2026.010.93
AttenMIA (Transitional)Model size=6.9b2026.010.89
PPL attackModel size=6.9b2026.010.88
MIN-K% PROBModel size=6.9b2026.010.88
AttenMIA (Transitional)Model size=1.4b2026.010.88
PPL attackModel size=2.8b2026.010.87
zlib attackModel size=6.9b2026.010.87
PETALModel size=6.9b2026.010.87
PPL attackModel size=1.4b2026.010.86
zlib attackModel size=2.8b2026.010.86
MIN-K% PROBModel size=1.4b2026.010.86
MIN-K% PROBModel size=2.8b2026.010.86
zlib attackModel size=1.4b2026.010.85
PETALModel size=1.4b2026.010.85
PETALModel size=2.8b2026.010.85
neighborhood attackModel size=6.9b2026.010.84
neighborhood attackModel size=1.4b2026.010.83
neighborhood attackModel size=2.8b2026.010.83
RS attackModel size=6.9b2026.010.78
WS attackModel size=6.9b2026.010.77
RS attackModel size=1.4b2026.010.77
RS attackModel size=2.8b2026.010.77
BT attackModel size=6.9b2026.010.77
WS attackModel size=1.4b2026.010.76
WS attackModel size=2.8b2026.010.76
BT attackModel size=2.8b2026.010.76
BT attackModel size=1.4b2026.010.74
RECALLModel size=6.9b2026.010.74
RECALLModel size=2.8b2026.010.72
RECALLModel size=1.4b2026.010.7
reference attackModel size=1.4b2026.010.65
reference attackModel size=2.8b2026.010.65
reference attackModel size=6.9b2026.010.64