Membership Inference Attack on WikiMIA length 128 (TPR@1%FPR)
98.84AUCReproMIA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ReproMIATarget Model=LLaMA-30B2026.03 | 98.84 | 59.02 | |
| ReproMIATarget Model=Pythia-6.9B2026.03 | 96.96 | 49.18 | |
| ReproMIATarget Model=LLaMA-13B2026.03 | 94.6 | 49.18 | |
| ReproMIATarget Model=OPT-66B2026.03 | 94.28 | 21.31 | |
| ReproMIATarget Model=Average2026.03 | 94.08 | 39.07 | |
| ReCaLLTarget Model=LLaMA-13B2026.03 | 90.67 | 19.67 | |
| ReproMIATarget Model=NeoX-20B2026.03 | 90.22 | 19.67 | |
| ReCaLLTarget Model=LLaMA-30B2026.03 | 90.08 | 18.03 | |
| ReproMIATarget Model=Mamba-1.4B2026.03 | 89.6 | 36.07 | |
| ReCaLLTarget Model=NeoX-20B2026.03 | 87.85 | 19.67 | |
| ReCaLLTarget Model=Mamba-1.4B2026.03 | 87.58 | 19.67 | |
| ReCaLLTarget Model=Pythia-6.9B2026.03 | 86.94 | 26.23 | |
| ReCaLLTarget Model=Average2026.03 | 86.06 | 18.31 | |
| Min-K%++Target Model=LLaMA-13B2026.03 | 84.33 | 12.68 | |
| Min-K%Target Model=LLaMA-30B2026.03 | 76.24 | 9.84 | |
| RefTarget Model=NeoX-20B2026.03 | 75.77 | 0 | |
| Min-K%++Target Model=NeoX-20B2026.03 | 75.4 | 1.64 | |
| ZlibTarget Model=LLaMA-30B2026.03 | 73.85 | 16.39 | |
| ReCaLLTarget Model=OPT-66B2026.03 | 73.26 | 6.56 | |
| Min-K%++Target Model=Average2026.03 | 72.84 | 6.9 | |
| Min-K%++Target Model=LLaMA-30B2026.03 | 72.61 | 8.2 | |
| RefTarget Model=LLaMA-30B2026.03 | 72.21 | 0 | |
| LossTarget Model=LLaMA-30B2026.03 | 72.13 | 8.2 | |
| Min-K%++Target Model=OPT-66B2026.03 | 71.85 | 8.2 | |
| Min-K%Target Model=NeoX-20B2026.03 | 71.3 | 3.28 | |
| Min-K%Target Model=LLaMA-13B2026.03 | 70.98 | 14.08 | |
| Min-K%Target Model=OPT-66B2026.03 | 70.71 | 3.28 | |
| Min-K%Target Model=Average2026.03 | 69.85 | 7.48 | |
| ZlibTarget Model=LLaMA-13B2026.03 | 69.29 | 15.49 | |
| RefTarget Model=OPT-66B2026.03 | 68.93 | 4.92 | |
| Min-K%++Target Model=Mamba-1.4B2026.03 | 68.71 | 2.47 | |
| NeighborTarget Model=LLaMA-30B2026.03 | 67.65 | — | |
| ZlibTarget Model=Average2026.03 | 67.64 | 9.97 | |
| ZlibTarget Model=NeoX-20B2026.03 | 67.37 | 4.92 | |
| RefTarget Model=Average2026.03 | 67.15 | 1.6 | |
| NeighborTarget Model=NeoX-20B2026.03 | 67 | — | |
| ZlibTarget Model=OPT-66B2026.03 | 67 | 6.56 | |
| LossTarget Model=LLaMA-13B2026.03 | 66.73 | 9.86 | |
| ZlibTarget Model=Mamba-1.4B2026.03 | 66.7 | 9.88 | |
| LossTarget Model=NeoX-20B2026.03 | 66.64 | 4.92 | |
| RefTarget Model=Mamba-1.4B2026.03 | 66.45 | 3.7 | |
| Min-K%Target Model=Mamba-1.4B2026.03 | 66.16 | 6.17 | |
| LossTarget Model=Average2026.03 | 65.84 | 5.13 | |
| LossTarget Model=OPT-66B2026.03 | 65.49 | 3.28 | |
| NeighborTarget Model=LLaMA-13B2026.03 | 64.21 | — | |
| Min-K%++Target Model=Pythia-6.9B2026.03 | 64.15 | 8.2 | |
| Min-K%Target Model=Pythia-6.9B2026.03 | 63.69 | 8.2 | |
| NeighborTarget Model=OPT-66B2026.03 | 63.45 | — | |
| NeighborTarget Model=Average2026.03 | 63.34 | — | |
| RefTarget Model=LLaMA-13B2026.03 | 63.26 | 0.98 | |
| LossTarget Model=Mamba-1.4B2026.03 | 63.19 | 1.23 | |
| NeighborTarget Model=Mamba-1.4B2026.03 | 62.2 | — | |
| ZlibTarget Model=Pythia-6.9B2026.03 | 61.65 | 6.56 | |
| LossTarget Model=Pythia-6.9B2026.03 | 60.83 | 3.28 | |
| RefTarget Model=Pythia-6.9B2026.03 | 56.29 | 0 | |
| NeighborTarget Model=Pythia-6.9B2026.03 | 55.55 | — |