Membership Inference Attack on WikiMIA length 128 (test)
96.72TPR @ 5% FPRReproMIA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ReproMIATarget Model=LLaMA-30B2026.03 | 96.72 | 55.74 | |
| ReproMIATarget Model=Pythia-6.9B2026.03 | 75.41 | 45.9 | |
| ReproMIATarget Model=LLaMA-13B2026.03 | 65.57 | 37.7 | |
| ReproMIATarget Model=Average2026.03 | 64.48 | 31.69 | |
| ReproMIATarget Model=Mamba-1.4B2026.03 | 50.82 | 18.03 | |
| ReproMIATarget Model=NeoX-20B2026.03 | 50.82 | 14.75 | |
| ReCaLLTarget Model=Mamba-1.4B2026.03 | 49.18 | 16.39 | |
| ReproMIATarget Model=OPT-66B2026.03 | 47.54 | 18.03 | |
| ReCaLLTarget Model=LLaMA-13B2026.03 | 45.9 | 19.67 | |
| ReCaLLTarget Model=NeoX-20B2026.03 | 45.9 | 11.27 | |
| Min-K%++Target Model=LLaMA-30B2026.03 | 42.62 | 8.2 | |
| Min-K%++Target Model=LLaMA-13B2026.03 | 42.25 | 8.45 | |
| ReCaLLTarget Model=Average2026.03 | 40.16 | 13.63 | |
| ReCaLLTarget Model=LLaMA-30B2026.03 | 37.7 | 1.64 | |
| ReCaLLTarget Model=Pythia-6.9B2026.03 | 36.07 | 26.23 | |
| Min-K%Target Model=LLaMA-30B2026.03 | 34.43 | 6.56 | |
| Min-K%Target Model=NeoX-20B2026.03 | 29.51 | 3.26 | |
| LossTarget Model=LLaMA-30B2026.03 | 27.87 | 8.2 | |
| Min-K%Target Model=OPT-66B2026.03 | 27.87 | 3.28 | |
| Min-K%++Target Model=Average2026.03 | 27.61 | 5.19 | |
| Min-K%++Target Model=NeoX-20B2026.03 | 26.23 | 5.04 | |
| ReCaLLTarget Model=OPT-66B2026.03 | 26.23 | 6.56 | |
| Min-K%Target Model=LLaMA-13B2026.03 | 25.35 | 14.08 | |
| Min-K%Target Model=Average2026.03 | 24.73 | 6.04 | |
| LossTarget Model=OPT-66B2026.03 | 22.95 | 3.28 | |
| ZlibTarget Model=LLaMA-30B2026.03 | 22.95 | 6.56 | |
| Min-K%++Target Model=OPT-66B2026.03 | 22.95 | 1.64 | |
| LossTarget Model=LLaMA-13B2026.03 | 22.54 | 9.86 | |
| ZlibTarget Model=OPT-66B2026.03 | 21.31 | 14.75 | |
| ZlibTarget Model=Mamba-1.4B2026.03 | 20.99 | 9.88 | |
| LossTarget Model=Average2026.03 | 20.16 | 4.05 | |
| ZlibTarget Model=LLaMA-13B2026.03 | 19.72 | 11.27 | |
| LossTarget Model=NeoX-20B2026.03 | 19.67 | 2.97 | |
| ZlibTarget Model=Average2026.03 | 19.35 | 8.39 | |
| Min-K%++Target Model=Pythia-6.9B2026.03 | 18.03 | 6.56 | |
| ZlibTarget Model=Pythia-6.9B2026.03 | 16.39 | 4.92 | |
| Min-K%Target Model=Pythia-6.9B2026.03 | 16.39 | 6.56 | |
| LossTarget Model=Mamba-1.4B2026.03 | 14.81 | 0 | |
| Min-K%Target Model=Mamba-1.4B2026.03 | 14.81 | 2.47 | |
| ZlibTarget Model=NeoX-20B2026.03 | 14.75 | 2.97 | |
| RefTarget Model=Mamba-1.4B2026.03 | 13.58 | 3.7 | |
| Min-K%++Target Model=Mamba-1.4B2026.03 | 13.58 | 1.23 | |
| LossTarget Model=Pythia-6.9B2026.03 | 13.11 | 0 | |
| RefTarget Model=NeoX-20B2026.03 | 13.11 | 0 | |
| RefTarget Model=LLaMA-30B2026.03 | 11.48 | 0 | |
| RefTarget Model=Average2026.03 | 9.96 | 1.44 | |
| RefTarget Model=LLaMA-13B2026.03 | 8.45 | 0 | |
| RefTarget Model=Pythia-6.9B2026.03 | 6.56 | 0 | |
| RefTarget Model=OPT-66B2026.03 | 6.56 | 4.92 |