Membership Inference on WikiMIA Hard 2024
0.64CoverageN-GRAM COVERAGE ATTACK
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| N-GRAM COVERAGE ATTACKModel=GPT-3.5 Inst.2025.08 | 0.64 | 0.63 | 0.61 | 0.61 | — | — | — | — | — | |
| N-GRAM COVERAGE ATTACKModel=LLaMa-65B2025.08 | 0.64 | 0.63 | 0.59 | 0.6 | — | — | — | — | — | |
| N-GRAM COVERAGE ATTACKModel=LLaMa-30B2025.08 | 0.61 | 0.61 | 0.55 | 0.57 | — | — | — | — | — | |
| N-GRAM COVERAGE ATTACKModel=GPT-3.5-01252025.08 | 0.59 | 0.56 | 0.54 | 0.55 | — | — | — | — | — | |
| N-GRAM COVERAGE ATTACKModel=LLaMa-13B2025.08 | 0.59 | 0.58 | 0.53 | 0.53 | — | — | — | — | — | |
| N-GRAM COVERAGE ATTACKModel=GPT-3.5-11062025.08 | 0.58 | 0.58 | 0.56 | 0.57 | — | — | — | — | — | |
| N-GRAM COVERAGE ATTACKModel=GPT-42025.08 | 0.57 | 0.58 | 0.55 | 0.57 | — | — | — | — | — | |
| N-GRAM COVERAGE ATTACKModel=GPT-4o-11202025.08 | 0.55 | 0.55 | 0.54 | 0.52 | — | — | — | — | — | |
| N-GRAM COVERAGE ATTACKModel=GPT-4o Mini2025.08 | 0.55 | 0.53 | 0.52 | 0.51 | — | — | — | — | — | |
| N-GRAM COVERAGE ATTACKModel=LLaMa-7B2025.08 | 0.55 | 0.54 | 0.53 | 0.52 | — | — | — | — | — | |
| DE-COPModel=GPT-3.5-01252025.08 | — | — | — | — | 0.47 | — | — | — | — | |
| DE-COPModel=GPT-3.5 Inst.2025.08 | — | — | — | — | 0.45 | — | — | — | — | |
| DE-COPModel=GPT-3.5-11062025.08 | — | — | — | — | 0.49 | — | — | — | — | |
| DE-COPModel=GPT-42025.08 | — | — | — | — | 0.44 | — | — | — | — | |
| DE-COPModel=GPT-4o-11202025.08 | — | — | — | — | 0.51 | — | — | — | — | |
| DE-COPModel=GPT-4o Mini2025.08 | — | — | — | — | 0.43 | — | — | — | — | |
| DE-COPModel=LLaMa-7B2025.08 | — | — | — | — | 0.47 | — | — | — | — | |
| DE-COPModel=LLaMa-13B2025.08 | — | — | — | — | 0.51 | — | — | — | — | |
| DE-COPModel=LLaMa-30B2025.08 | — | — | — | — | 0.5 | — | — | — | — | |
| DE-COPModel=LLaMa-65B2025.08 | — | — | — | — | 0.51 | — | — | — | — | |
| LossModel=LLaMa-7B2025.08 | — | — | — | — | — | 0.51 | — | — | — | |
| LossModel=LLaMa-13B2025.08 | — | — | — | — | — | 0.53 | — | — | — | |
| LossModel=LLaMa-30B2025.08 | — | — | — | — | — | 0.56 | — | — | — | |
| LossModel=LLaMa-65B2025.08 | — | — | — | — | — | 0.57 | — | — | — | |
| Min-K%Model=LLaMa-7B2025.08 | — | — | — | — | — | — | — | — | 0.52 | |
| Min-K%Model=LLaMa-13B2025.08 | — | — | — | — | — | — | — | — | 0.54 | |
| Min-K%Model=LLaMa-30B2025.08 | — | — | — | — | — | — | — | — | 0.6 | |
| Min-K%Model=LLaMa-65B2025.08 | — | — | — | — | — | — | — | — | 0.58 | |
| Reference LossModel=LLaMa-13B2025.08 | — | — | — | — | — | — | 0.57 | — | — | |
| Reference LossModel=LLaMa-30B2025.08 | — | — | — | — | — | — | 0.61 | — | — | |
| Reference LossModel=LLaMa-65B2025.08 | — | — | — | — | — | — | 0.57 | — | — | |
| zlibModel=LLaMa-7B2025.08 | — | — | — | — | — | — | — | 0.5 | — | |
| zlibModel=LLaMa-13B2025.08 | — | — | — | — | — | — | — | 0.51 | — | |
| zlibModel=LLaMa-30B2025.08 | — | — | — | — | — | — | — | 0.53 | — | |
| zlibModel=LLaMa-65B2025.08 | — | — | — | — | — | — | — | 0.54 | — |