Membership Inference Attack on WikiMIA length 128
0.874AUCReCaLL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ReCaLLModel=Pythia-6.9B, Setting=Gray-Box2026.01 | 0.874 | 41.3 | |
| Min-K%++Model=LLaMA-13B, Setting=Gray-Box2026.01 | 0.864 | 38.5 | |
| SimMIA*Model=Average, Setting=Black-Box, Scoring=Equation 12026.01 | 0.794 | 21.9 | |
| Min-K% ProbTarget LLM=GPT-J-6B2024.04 | 0.7 | — | |
| SaMIA*zlibTarget LLM=OPT-6.7B2024.04 | 0.7 | — | |
| Min-K% ProbTarget LLM=Pythia-6.9B2024.04 | 0.69 | — | |
| PPL/zlibTarget LLM=GPT-J-6B2024.04 | 0.68 | — | |
| SaMIA*zlibTarget LLM=Pythia-6.9B2024.04 | 0.68 | — | |
| SaMIAModel=GPT-NeoX-20B, Setting=Black-Box2026.01 | 0.675 | 16.3 | |
| LOSSTarget LLM=GPT-J-6B2024.04 | 0.67 | — | |
| Min-K% ProbTarget LLM=OPT-6.7B2024.04 | 0.67 | — | |
| PPL/zlibTarget LLM=Pythia-6.9B2024.04 | 0.67 | — | |
| SaMIA*zlibTarget LLM=GPT-J-6B2024.04 | 0.66 | — | |
| SaMIATarget LLM=OPT-6.7B2024.04 | 0.66 | — | |
| SaMIATarget LLM=Pythia-6.9B2024.04 | 0.66 | — | |
| LOSSTarget LLM=Pythia-6.9B2024.04 | 0.65 | — | |
| PPL/zlibTarget LLM=OPT-6.7B2024.04 | 0.64 | — | |
| SaMIATarget LLM=GPT-J-6B2024.04 | 0.63 | — | |
| SaMIA*zlibTarget LLM=LLAMA-2-7B2024.04 | 0.63 | — | |
| LOSSTarget LLM=OPT-6.7B2024.04 | 0.62 | — | |
| SaMIATarget LLM=LLAMA-2-7B2024.04 | 0.6 | — | |
| LowercaseTarget LLM=GPT-J-6B2024.04 | 0.58 | — | |
| LowercaseTarget LLM=OPT-6.7B2024.04 | 0.57 | — | |
| LowercaseTarget LLM=Pythia-6.9B2024.04 | 0.57 | — | |
| PPL/zlibTarget LLM=LLAMA-2-7B2024.04 | 0.57 | — | |
| LOSSTarget LLM=LLAMA-2-7B2024.04 | 0.56 | — | |
| Min-K% ProbTarget LLM=LLAMA-2-7B2024.04 | 0.56 | — | |
| LowercaseTarget LLM=LLAMA-2-7B2024.04 | 0.49 | — |