Membership Inference on WikiMIA 32 tokens 1.0
99.7ROC AUCAttenMIA (Transitional)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AttenMIA (Transitional)Target Model=LLaMA-30B2026.01 | 99.7 | 94.1 | |
| AttenMIA (Perturbed)Target Model=LLaMA-30B2026.01 | 99.7 | 94.7 | |
| AttenMIA (Perturbed)Target Model=LLaMA-13B2026.01 | 99.6 | 87.9 | |
| AttenMIA (Transitional)Target Model=NeoX-20B2026.01 | 99.4 | 86.8 | |
| AttenMIA (Transitional)Target Model=LLaMA-13B2026.01 | 99.3 | 84.8 | |
| AttenMIA (Transitional)Target Model=Average2026.01 | 99 | 87.3 | |
| AttenMIA (Transitional)Target Model=Pythia-6.9B2026.01 | 98.9 | 86.4 | |
| AttenMIA (Perturbed)Target Model=NeoX-20B2026.01 | 98.9 | 86.4 | |
| AttenMIA (Perturbed)Target Model=Average2026.01 | 98.8 | 83.9 | |
| AttenMIA (Perturbed)Target Model=Pythia-6.9B2026.01 | 98.2 | 66 | |
| AttenMIA (Perturbed)Target Model=OPT-66B2026.01 | 97.8 | 84.6 | |
| AttenMIA (Transitional)Target Model=OPT-66B2026.01 | 97.6 | 84.2 | |
| RECALLTarget Model=LLaMA-13B2026.01 | 92.2 | 13.3 | |
| RECALLTarget Model=Pythia-6.9B2026.01 | 91.6 | 28.5 | |
| RECALLTarget Model=LLaMA-30B2026.01 | 90.7 | 18.4 | |
| RECALLTarget Model=NeoX-20B2026.01 | 90.5 | 25.3 | |
| RECALLTarget Model=Average2026.01 | 90.1 | 17.5 | |
| RECALLTarget Model=OPT-66B2026.01 | 85.1 | 8.3 | |
| Min-K%++Target Model=LLaMA-13B2026.01 | 84.8 | 10.4 | |
| Min-K%++Target Model=LLaMA-30B2026.01 | 84.3 | 9.3 | |
| Min-K%++Target Model=Average2026.01 | 75.3 | 6.6 | |
| Min-K%++Target Model=NeoX-20B2026.01 | 75.1 | 6.1 | |
| Min-K%Target Model=NeoX-20B2026.01 | 71.8 | 10.7 | |
| PPLTarget Model=LLaMA-30B2026.01 | 71.1 | 4.9 | |
| PPLTarget Model=NeoX-20B2026.01 | 71 | 2.1 | |
| Min-K%++Target Model=Pythia-6.9B2026.01 | 70.3 | 5.9 | |
| Min-K%++Target Model=OPT-66B2026.01 | 70.3 | 3.7 | |
| NeighborTarget Model=NeoX-20B2026.01 | 70.2 | 0.6 | |
| Min-K%Target Model=LLaMA-30B2026.01 | 70.1 | 4.5 | |
| RefTarget Model=LLaMA-30B2026.01 | 70 | 2.7 | |
| ZlibTarget Model=LLaMA-30B2026.01 | 69.9 | 4.8 | |
| LossTarget Model=LLaMA-30B2026.01 | 69.5 | 4.3 | |
| RefTarget Model=NeoX-20B2026.01 | 68.9 | 10.1 | |
| ZlibTarget Model=NeoX-20B2026.01 | 68.9 | 9.1 | |
| LossTarget Model=NeoX-20B2026.01 | 68.7 | 10.4 | |
| PPLTarget Model=Average2026.01 | 68.5 | 3.6 | |
| PPLTarget Model=OPT-66B2026.01 | 68.3 | 3.4 | |
| PPLTarget Model=LLaMA-13B2026.01 | 68.2 | 1.3 | |
| NeighborTarget Model=OPT-66B2026.01 | 68.2 | 0.8 | |
| Min-K%Target Model=LLaMA-13B2026.01 | 68 | 5.1 | |
| ZlibTarget Model=LLaMA-13B2026.01 | 67.8 | 5.6 | |
| Min-K%Target Model=Average2026.01 | 67.8 | 7.5 | |
| RefTarget Model=LLaMA-13B2026.01 | 67.7 | 5.9 | |
| LossTarget Model=LLaMA-13B2026.01 | 67.6 | 4.8 | |
| NeighborTarget Model=LLaMA-30B2026.01 | 67.6 | 0.6 | |
| Min-K%Target Model=OPT-66B2026.01 | 67.4 | 9.1 | |
| NeighborTarget Model=Average2026.01 | 66.9 | 0.7 | |
| ZlibTarget Model=Average2026.01 | 66.3 | 5.7 | |
| Min-K%Target Model=Pythia-6.9B2026.01 | 66.3 | 8.8 | |
| RefTarget Model=Average2026.01 | 66.2 | 6.1 | |
| LossTarget Model=Average2026.01 | 65.9 | 6.1 | |
| RefTarget Model=OPT-66B2026.01 | 65.8 | 6.7 | |
| NeighborTarget Model=Pythia-6.9B2026.01 | 65.8 | 0.8 | |
| NeighborTarget Model=LLaMA-13B2026.01 | 65.8 | 0.6 | |
| ZlibTarget Model=OPT-66B2026.01 | 65.5 | 5.6 | |
| LossTarget Model=OPT-66B2026.01 | 65.4 | 6.4 | |
| PPLTarget Model=Pythia-6.9B2026.01 | 64.1 | 6.2 | |
| ZlibTarget Model=Pythia-6.9B2026.01 | 64.1 | 4.8 | |
| PETALTarget Model=Pythia-6.9B2026.01 | 64.1 | 4.9 | |
| RefTarget Model=Pythia-6.9B2026.01 | 63.7 | 6.9 | |
| LossTarget Model=Pythia-6.9B2026.01 | 63.6 | 6.1 | |
| PETALTarget Model=NeoX-20B2026.01 | 62.1 | 2.1 | |
| PETALTarget Model=OPT-66B2026.01 | 62.1 | 3.1 | |
| PETALTarget Model=Average2026.01 | 61.3 | 2.7 | |
| PETALTarget Model=LLaMA-30B2026.01 | 60 | 1.9 | |
| PETALTarget Model=LLaMA-13B2026.01 | 58 | 1.6 |