Membership Inference Attack on SST-2
0.999ROC AUCRMIA
Evaluation results
| Method | Links | |
|---|---|---|
| RMIAThreat Model=Model-based (A^theta), Source=Synthetic, Label=Natural, Canary Length=50 words2025.02 | 0.999 | |
| RMIAThreat Model=Model-based (A^theta), Source=Synthetic, Label=Artificial, Canary Length=50 words2025.02 | 0.999 | |
| RMIAThreat Model=Model-based (A^theta), Source=In-distribution, Canary Length=30 words2025.02 | 0.911 | |
| n-gram MIAThreat Model=Data-based (A^D), Methodology=2-gram, Source=In-distribution, Canary Length=30 words2025.02 | 0.741 | |
| n-gram MIAThreat Model=Data-based (A^D), Methodology=2-gram, Source=Synthetic, Label=Artificial, Canary Length=50 words2025.02 | 0.682 | |
| n-gram MIAThreat Model=Data-based (A^D), Methodology=2-gram, Source=Synthetic, Label=Natural, Canary Length=50 words2025.02 | 0.62 | |
| similarity-based MIA (SIMJac)Threat Model=Data-based (A^D), Methodology=SIMJac, Source=In-distribution, Canary Length=30 words2025.02 | 0.602 | |
| similarity-based MIA (SIMemb)Threat Model=Data-based (A^D), Methodology=SIMemb, Source=In-distribution, Canary Length=30 words2025.02 | 0.586 | |
| similarity-based MIA (SIMJac)Threat Model=Data-based (A^D), Methodology=SIMJac, Source=Synthetic, Label=Artificial, Canary Length=50 words2025.02 | 0.552 | |
| similarity-based MIA (SIMJac)Threat Model=Data-based (A^D), Methodology=SIMJac, Source=Synthetic, Label=Natural, Canary Length=50 words2025.02 | 0.547 | |
| similarity-based MIA (SIMemb)Threat Model=Data-based (A^D), Methodology=SIMemb, Source=Synthetic, Label=Artificial, Canary Length=50 words2025.02 | 0.539 | |
| similarity-based MIA (SIMemb)Threat Model=Data-based (A^D), Methodology=SIMemb, Source=Synthetic, Label=Natural, Canary Length=50 words2025.02 | 0.53 |