Membership Inference Attack on WikiMIA
71AUCSaMIA*zlib
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| SaMIA*zlibTarget LLM=OPT-6.7B, Likelihood dependency=likelihood-independent2024.04 | 71 | 37.3 | |
| Min-K% ProbTarget LLM=GPT-J-6B, Likelihood dependency=likelihood-dependent2024.04 | 69 | 26.8 | |
| Min-K% ProbTarget LLM=Pythia-6.9B, Likelihood dependency=likelihood-dependent2024.04 | 68 | 26.6 | |
| LOSSTarget LLM=GPT-J-6B, Likelihood dependency=likelihood-dependent2024.04 | 66 | 18 | |
| PPL/zlibTarget LLM=GPT-J-6B, Likelihood dependency=likelihood-dependent2024.04 | 66 | 21.3 | |
| SaMIA*zlibTarget LLM=GPT-J-6B, Likelihood dependency=likelihood-independent2024.04 | 66 | 26 | |
| SaMIATarget LLM=OPT-6.7B, Likelihood dependency=likelihood-independent2024.04 | 66 | 31.8 | |
| PPL/zlibTarget LLM=Pythia-6.9B, Likelihood dependency=likelihood-dependent2024.04 | 66 | 19.6 | |
| SaMIA*zlibTarget LLM=Pythia-6.9B, Likelihood dependency=likelihood-independent2024.04 | 66 | 23.7 | |
| LOSSTarget LLM=Pythia-6.9B, Likelihood dependency=likelihood-dependent2024.04 | 65 | 20.4 | |
| SaMIATarget LLM=GPT-J-6B, Likelihood dependency=likelihood-independent2024.04 | 64 | 19.9 | |
| Min-K% ProbTarget LLM=OPT-6.7B, Likelihood dependency=likelihood-dependent2024.04 | 64 | 25.8 | |
| SaMIATarget LLM=Pythia-6.9B, Likelihood dependency=likelihood-independent2024.04 | 64 | 20.4 | |
| PPL/zlibTarget LLM=OPT-6.7B, Likelihood dependency=likelihood-dependent2024.04 | 62 | 17.5 | |
| SaMIA*zlibTarget LLM=LLAMA-2-7B, Likelihood dependency=likelihood-independent2024.04 | 62 | 20 | |
| LOSSTarget LLM=OPT-6.7B, Likelihood dependency=likelihood-dependent2024.04 | 61 | 16.8 | |
| SaMIATarget LLM=LLAMA-2-7B, Likelihood dependency=likelihood-independent2024.04 | 60 | 16.4 | |
| LowercaseTarget LLM=GPT-J-6B, Likelihood dependency=likelihood-dependent2024.04 | 59 | 16.7 | |
| LowercaseTarget LLM=OPT-6.7B, Likelihood dependency=likelihood-dependent2024.04 | 58 | 16 | |
| LowercaseTarget LLM=Pythia-6.9B, Likelihood dependency=likelihood-dependent2024.04 | 57 | 17 | |
| PPL/zlibTarget LLM=LLAMA-2-7B, Likelihood dependency=likelihood-dependent2024.04 | 56 | 16.1 | |
| LOSSTarget LLM=LLAMA-2-7B, Likelihood dependency=likelihood-dependent2024.04 | 55 | 13.8 | |
| Min-K% ProbTarget LLM=LLAMA-2-7B, Likelihood dependency=likelihood-dependent2024.04 | 54 | 10.7 | |
| LowercaseTarget LLM=LLAMA-2-7B, Likelihood dependency=likelihood-dependent2024.04 | 52 | 15.5 |