Membership Inference on DM Math Pythia (train)
99.3TPR@1%FPRAttenMIA (Perturbed)
Evaluation Results
| Method | Links | |
|---|---|---|
| AttenMIA (Perturbed)Model Size=6.9b2026.01 | 99.3 | |
| AttenMIA (Perturbed)Model Size=1.4b2026.01 | 98.9 | |
| AttenMIA (Perturbed)Model Size=2.8b2026.01 | 98.8 | |
| AttenMIA (Transitional)Model Size=2.8b2026.01 | 87.8 | |
| AttenMIA (Transitional)Model Size=1.4b2026.01 | 86.7 | |
| AttenMIA (Transitional)Model Size=6.9b2026.01 | 85.6 | |
| PPL attackModel Size=1.4b2026.01 | 51.7 | |
| reference attackModel Size=2.8b2026.01 | 50.6 | |
| reference attackModel Size=1.4b2026.01 | 49.4 | |
| reference attackModel Size=6.9b2026.01 | 47.2 | |
| RECALLModel Size=1.4b2026.01 | 44.2 | |
| PPL attackModel Size=6.9b2026.01 | 42.7 | |
| PPL attackModel Size=2.8b2026.01 | 34.8 | |
| RECALLModel Size=2.8b2026.01 | 31.2 | |
| RECALLModel Size=6.9b2026.01 | 22.1 |