Membership Inference Attack on AG News, WikiText-103, XSum, and Swallow-Code (held-out)
0.585AUCMin-K%++
Evaluation Results
| Method | Links | |
|---|---|---|
| Min-K%++Target Architecture=Transformer2026.04 | 0.585 | |
| ZlibTarget Architecture=Transformer2026.04 | 0.611 | |
| LossTarget Architecture=Transformer2026.04 | 0.622 | |
| Min-K%++Target Architecture=RWKV2026.04 | 0.706 | |
| Min-K%++Target Architecture=Mean2026.04 | 0.73 | |
| RefLossTarget Architecture=Transformer2026.04 | 0.76 | |
| ZlibTarget Architecture=RWKV2026.04 | 0.761 | |
| ZlibTarget Architecture=Mean2026.04 | 0.761 | |
| LossTarget Architecture=Mean2026.04 | 0.784 | |
| LossTarget Architecture=RWKV2026.04 | 0.786 | |
| RefLossTarget Architecture=RecurrentGemma2026.04 | 0.8 | |
| Min-K%++Target Architecture=RecurrentGemma2026.04 | 0.813 | |
| Min-K%++Target Architecture=Mamba2026.04 | 0.814 | |
| ZlibTarget Architecture=RecurrentGemma2026.04 | 0.836 | |
| ZlibTarget Architecture=Mamba2026.04 | 0.837 | |
| RefLossTarget Architecture=Mean2026.04 | 0.842 | |
| LossTarget Architecture=RecurrentGemma2026.04 | 0.86 | |
| LossTarget Architecture=Mamba2026.04 | 0.867 | |
| EZ-MIATarget Architecture=Transformer2026.04 | 0.887 | |
| RefLossTarget Architecture=Mamba2026.04 | 0.892 | |
| LT-MIATarget Architecture=Transformer2026.04 | 0.908 | |
| RefLossTarget Architecture=RWKV2026.04 | 0.915 | |
| EZ-MIATarget Architecture=RecurrentGemma2026.04 | 0.915 | |
| EZ-MIATarget Architecture=Mean2026.04 | 0.926 | |
| EZ-MIATarget Architecture=Mamba2026.04 | 0.931 | |
| LT-MIATarget Architecture=RecurrentGemma2026.04 | 0.936 | |
| LT-MIATarget Architecture=Mean2026.04 | 0.945 | |
| LT-MIATarget Architecture=Mamba2026.04 | 0.963 | |
| EZ-MIATarget Architecture=RWKV2026.04 | 0.971 | |
| LT-MIATarget Architecture=RWKV2026.04 | 0.972 |