Membership Inference Attack on WikiMIA length 64 (test)
73.77TPR@5%FPRReproMIA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ReproMIATarget Model=LLaMA-13B2026.03 | 73.77 | 32.79 | |
| ReproMIATarget Model=Pythia-6.9B2026.03 | 70.67 | 25 | |
| ReproMIATarget Model=LLaMA-30B2026.03 | 67.31 | 9.62 | |
| ReCaLLTarget Model=LLaMA-30B2026.03 | 62.02 | 6.73 | |
| ReCaLLTarget Model=LLaMA-13B2026.03 | 61.06 | 25.48 | |
| ReproMIATarget Model=Average2026.03 | 58.53 | 15.26 | |
| ReproMIATarget Model=Mamba-1.4B2026.03 | 57.69 | 11.06 | |
| ReCaLLTarget Model=Pythia-6.9B2026.03 | 54.81 | 1.44 | |
| ReCaLLTarget Model=Mamba-1.4B2026.03 | 53.37 | 9.13 | |
| ReCaLLTarget Model=Average2026.03 | 49.25 | 8.09 | |
| ReproMIATarget Model=OPT-66B2026.03 | 46.63 | 8.17 | |
| ReCaLLTarget Model=NeoX-20B2026.03 | 42.62 | 3.37 | |
| ReproMIATarget Model=NeoX-20B2026.03 | 35.1 | 4.91 | |
| Min-K%++Target Model=LLaMA-13B2026.03 | 34.13 | 4.81 | |
| Min-K%++Target Model=LLaMA-30B2026.03 | 32.21 | 0.96 | |
| Min-K%++Target Model=OPT-66B2026.03 | 28.85 | 1.44 | |
| Min-K%++Target Model=Average2026.03 | 25.96 | 4.01 | |
| Min-K%++Target Model=Pythia-6.9B2026.03 | 25.48 | 4.33 | |
| Min-K%Target Model=NeoX-20B2026.03 | 22.12 | 3.85 | |
| Min-K%Target Model=Pythia-6.9B2026.03 | 21.63 | 3.37 | |
| Min-K%Target Model=LLaMA-30B2026.03 | 21.63 | 1.92 | |
| Min-K%Target Model=OPT-66B2026.03 | 21.63 | 5.77 | |
| ReCaLLTarget Model=OPT-66B2026.03 | 21.63 | 2.4 | |
| Min-K%++Target Model=NeoX-20B2026.03 | 21.15 | 5.77 | |
| RefTarget Model=NeoX-20B2026.03 | 20.67 | 3.85 | |
| Min-K%Target Model=Average2026.03 | 20.59 | 3.69 | |
| Min-K%Target Model=LLaMA-13B2026.03 | 19.23 | 3.85 | |
| ZlibTarget Model=NeoX-20B2026.03 | 17.79 | 2.88 | |
| Min-K%Target Model=Mamba-1.4B2026.03 | 17.31 | 3.37 | |
| ZlibTarget Model=Pythia-6.9B2026.03 | 15.38 | 1.44 | |
| ZlibTarget Model=Average2026.03 | 15.06 | 2.97 | |
| LossTarget Model=Pythia-6.9B2026.03 | 14.9 | 0.96 | |
| ZlibTarget Model=Mamba-1.4B2026.03 | 14.9 | 2.4 | |
| ZlibTarget Model=LLaMA-30B2026.03 | 14.42 | 3.85 | |
| ZlibTarget Model=OPT-66B2026.03 | 14.42 | 3.85 | |
| Min-K%++Target Model=Mamba-1.4B2026.03 | 13.94 | 6.73 | |
| ZlibTarget Model=LLaMA-13B2026.03 | 13.46 | 3.37 | |
| LossTarget Model=LLaMA-30B2026.03 | 12.5 | 1.92 | |
| LossTarget Model=OPT-66B2026.03 | 12.5 | 1.92 | |
| LossTarget Model=Average2026.03 | 12.18 | 1.6 | |
| LossTarget Model=NeoX-20B2026.03 | 12.02 | 1.92 | |
| LossTarget Model=Mamba-1.4B2026.03 | 10.58 | 0.96 | |
| LossTarget Model=LLaMA-13B2026.03 | 10.58 | 1.92 | |
| RefTarget Model=Pythia-6.9B2026.03 | 10.58 | 0 | |
| RefTarget Model=LLaMA-30B2026.03 | 10.1 | 0 | |
| RefTarget Model=OPT-66B2026.03 | 10.1 | 0 | |
| RefTarget Model=Average2026.03 | 9.86 | 0.72 | |
| RefTarget Model=Mamba-1.4B2026.03 | 4.33 | 0.48 | |
| RefTarget Model=LLaMA-13B2026.03 | 3.37 | 0 |