Membership Inference Attack on WikiMIA length 64
0.9556AUCReproMIA
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| ReproMIATarget Model=Pythia-6.9B2026.03 | 0.9556 | — | 36.06 | |
| ReproMIATarget Model=LLaMA-30B2026.03 | 0.952 | — | 35.1 | |
| ReproMIATarget Model=LLaMA-13B2026.03 | 0.9417 | — | 37.7 | |
| ReCaLLTarget Model=LLaMA-30B2026.03 | 0.9395 | — | 26.64 | |
| ReproMIATarget Model=OPT-66B2026.03 | 0.9314 | — | 16.83 | |
| ReCaLLTarget Model=LLaMA-13B2026.03 | 0.9247 | — | 28.85 | |
| ReCaLLTarget Model=Pythia-6.9B2026.03 | 0.9237 | — | 24.52 | |
| ReproMIATarget Model=Average2026.03 | 0.9221 | — | 25.52 | |
| ReCaLLTarget Model=NeoX-20B2026.03 | 0.9087 | — | 7.21 | |
| ReCaLLTarget Model=Average2026.03 | 0.9059 | — | 18.62 | |
| ReproMIATarget Model=Mamba-1.4B2026.03 | 0.9038 | — | 16.83 | |
| ReCaLLTarget Model=Mamba-1.4B2026.03 | 0.9018 | — | 13.94 | |
| SimMIA*Model=Pythia-6.9B, Setting=Black-Box, Scoring=Equation 12026.01 | 0.875 | 37.5 | — | |
| Min-K%++Target Model=LLaMA-13B2026.03 | 0.8671 | — | 14.42 | |
| SimMIA*Model=OPT-6.7B, Setting=Black-Box, Scoring=Equation 12026.01 | 0.86 | 25.1 | — | |
| ReCaLLModel=Average, Setting=Gray-Box2026.01 | 0.857 | 39 | — | |
| Min-K%++Target Model=LLaMA-30B2026.03 | 0.8535 | — | 7.69 | |
| Min-K%++Target Model=OPT-66B2026.03 | 0.8535 | — | 8.17 | |
| ReproMIATarget Model=NeoX-20B2026.03 | 0.8482 | — | 10.58 | |
| ReCaLLTarget Model=OPT-66B2026.03 | 0.8369 | — | 10.58 | |
| Min-K%++Target Model=Average2026.03 | 0.7877 | — | 9.86 | |
| Min-K%++Target Model=NeoX-20B2026.03 | 0.7567 | — | 9.13 | |
| RefTarget Model=NeoX-20B2026.03 | 0.7314 | — | 8.17 | |
| Min-K%++Target Model=Pythia-6.9B2026.03 | 0.7268 | — | 11.55 | |
| Min-K%Target Model=NeoX-20B2026.03 | 0.715 | — | 7.21 | |
| RefTarget Model=LLaMA-30B2026.03 | 0.6934 | — | 1.44 | |
| RefTarget Model=OPT-66B2026.03 | 0.6934 | — | 1.44 | |
| ZlibTarget Model=NeoX-20B2026.03 | 0.6866 | — | 6.73 | |
| Min-K%Target Model=LLaMA-30B2026.03 | 0.6841 | — | 8.65 | |
| Min-K%Target Model=OPT-66B2026.03 | 0.6841 | — | 8.65 | |
| SaMIA*zlibTarget LLM=OPT-6.7B2024.04 | 0.68 | — | — | |
| ZlibTarget Model=LLaMA-30B2026.03 | 0.6755 | — | 9.13 | |
| ZlibTarget Model=OPT-66B2026.03 | 0.6755 | — | 9.13 | |
| Min-K%Target Model=Average2026.03 | 0.6707 | — | 9.21 | |
| Min-K%++Target Model=Mamba-1.4B2026.03 | 0.6684 | — | 8.17 | |
| RefTarget Model=Average2026.03 | 0.6677 | — | 2.56 | |
| NeighborTarget Model=NeoX-20B2026.03 | 0.6667 | — | — | |
| LossTarget Model=NeoX-20B2026.03 | 0.6656 | — | 4.81 | |
| Min-K%Target Model=LLaMA-13B2026.03 | 0.6646 | — | 7.21 | |
| LossTarget Model=LLaMA-30B2026.03 | 0.6612 | — | 5.29 | |
| LossTarget Model=OPT-66B2026.03 | 0.6612 | — | 5.29 | |
| Min-K% ProbTarget LLM=GPT-J-6B2024.04 | 0.66 | — | — | |
| Min-K%Target Model=Pythia-6.9B2026.03 | 0.6597 | — | 12.02 | |
| ZlibTarget Model=LLaMA-13B2026.03 | 0.6577 | — | 7.69 | |
| ZlibTarget Model=Average2026.03 | 0.6534 | — | 7.93 | |
| ZlibModel=LLaMA-13B, Setting=Gray-Box2026.01 | 0.652 | 12.7 | — | |
| LossTarget Model=LLaMA-13B2026.03 | 0.6417 | — | 3.85 | |
| Min-K% ProbTarget LLM=Pythia-6.9B2024.04 | 0.64 | — | — | |
| RefTarget Model=LLaMA-13B2026.03 | 0.6371 | — | 2.88 | |
| LossTarget Model=Average2026.03 | 0.636 | — | 3.93 | |
| ZlibTarget Model=Pythia-6.9B2026.03 | 0.6321 | — | 9.62 | |
| PPL/zlibTarget LLM=GPT-J-6B2024.04 | 0.63 | — | — | |
| SaMIATarget LLM=OPT-6.7B2024.04 | 0.63 | — | — | |
| SaMIA*zlibTarget LLM=Pythia-6.9B2024.04 | 0.63 | — | — | |
| RefTarget Model=Pythia-6.9B2026.03 | 0.6256 | — | 0.96 | |
| RefTarget Model=Mamba-1.4B2026.03 | 0.6254 | — | 0.48 | |
| NeighborTarget Model=LLaMA-13B2026.03 | 0.6232 | — | — | |
| NeighborTarget Model=LLaMA-30B2026.03 | 0.6209 | — | — | |
| NeighborTarget Model=OPT-66B2026.03 | 0.6209 | — | — | |
| NeighborTarget Model=Average2026.03 | 0.6207 | — | — | |
| LOSSTarget LLM=GPT-J-6B2024.04 | 0.62 | — | — | |
| SaMIA*zlibTarget LLM=GPT-J-6B2024.04 | 0.62 | — | — | |
| PPL/zlibTarget LLM=Pythia-6.9B2024.04 | 0.62 | — | — | |
| Min-K%Target Model=Mamba-1.4B2026.03 | 0.6165 | — | 11.54 | |
| LossTarget Model=Pythia-6.9B2026.03 | 0.6162 | — | 1.92 | |
| LOSSTarget LLM=Pythia-6.9B2024.04 | 0.61 | — | — | |
| SaMIATarget LLM=Pythia-6.9B2024.04 | 0.61 | — | — | |
| NeighborTarget Model=Pythia-6.9B2026.03 | 0.606 | — | — | |
| SaMIATarget LLM=GPT-J-6B2024.04 | 0.6 | — | — | |
| Min-K% ProbTarget LLM=OPT-6.7B2024.04 | 0.6 | — | — | |
| ZlibTarget Model=Mamba-1.4B2026.03 | 0.5931 | — | 5.29 | |
| SaMIA*zlibTarget LLM=LLAMA-2-7B2024.04 | 0.59 | — | — | |
| NeighborTarget Model=Mamba-1.4B2026.03 | 0.5862 | — | — | |
| PPL/zlibTarget LLM=OPT-6.7B2024.04 | 0.58 | — | — | |
| LowercaseTarget LLM=GPT-J-6B2024.04 | 0.57 | — | — | |
| LOSSTarget LLM=OPT-6.7B2024.04 | 0.57 | — | — | |
| LowercaseTarget LLM=OPT-6.7B2024.04 | 0.57 | — | — | |
| LossTarget Model=Mamba-1.4B2026.03 | 0.5698 | — | 2.4 | |
| LowercaseTarget LLM=Pythia-6.9B2024.04 | 0.55 | — | — | |
| SaMIATarget LLM=LLAMA-2-7B2024.04 | 0.55 | — | — | |
| PPL/zlibTarget LLM=LLAMA-2-7B2024.04 | 0.51 | — | — | |
| LOSSTarget LLM=LLAMA-2-7B2024.04 | 0.5 | — | — | |
| LowercaseTarget LLM=LLAMA-2-7B2024.04 | 0.5 | — | — | |
| Min-K% ProbTarget LLM=LLAMA-2-7B2024.04 | 0.5 | — | — |