Membership Inference Attack on The Pile (test)
0.54CoverageN-GRAM COVERAGE ATTACK
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| N-GRAM COVERAGE ATTACKModel=Pythia 2.8B, Attack Category=Output-Only2025.08 | 0.54 | 0.54 | — | — | — | — | — | — | — | |
| N-GRAM COVERAGE ATTACKModel=Pythia 12B, Attack Category=Output-Only2025.08 | 0.54 | 0.54 | — | — | — | — | — | — | — | |
| N-GRAM COVERAGE ATTACKModel=Pythia 1.4B, Attack Category=Output-Only2025.08 | 0.53 | 0.53 | — | — | — | — | — | — | — | |
| N-GRAM COVERAGE ATTACKModel=Pythia 6.9B, Attack Category=Output-Only2025.08 | 0.53 | 0.53 | — | — | — | — | — | — | — | |
| D-CModel=Pythia 1.4B, Attack Category=Output-Only2025.08 | — | — | — | — | 0.5 | — | — | — | — | |
| D-CModel=Pythia 2.8B, Attack Category=Output-Only2025.08 | — | — | — | — | 0.5 | — | — | — | — | |
| D-CModel=Pythia 6.9B, Attack Category=Output-Only2025.08 | — | — | — | — | 0.5 | — | — | — | — | |
| D-CModel=Pythia 12B, Attack Category=Output-Only2025.08 | — | — | — | — | 0.5 | — | — | — | — | |
| LCScModel=Pythia 1.4B, Attack Category=Output-Only2025.08 | — | — | 0.51 | — | — | — | — | — | — | |
| LCScModel=Pythia 2.8B, Attack Category=Output-Only2025.08 | — | — | 0.49 | — | — | — | — | — | — | |
| LCScModel=Pythia 6.9B, Attack Category=Output-Only2025.08 | — | — | 0.5 | — | — | — | — | — | — | |
| LCScModel=Pythia 12B, Attack Category=Output-Only2025.08 | — | — | 0.52 | — | — | — | — | — | — | |
| LCSwModel=Pythia 1.4B, Attack Category=Output-Only2025.08 | — | — | — | 0.52 | — | — | — | — | — | |
| LCSwModel=Pythia 2.8B, Attack Category=Output-Only2025.08 | — | — | — | 0.5 | — | — | — | — | — | |
| LCSwModel=Pythia 6.9B, Attack Category=Output-Only2025.08 | — | — | — | 0.51 | — | — | — | — | — | |
| LCSwModel=Pythia 12B, Attack Category=Output-Only2025.08 | — | — | — | 0.51 | — | — | — | — | — | |
| LossModel=Pythia 1.4B, Attack Category=Loss-Based2025.08 | — | — | — | — | — | 0.54 | — | — | — | |
| LossModel=Pythia 2.8B, Attack Category=Loss-Based2025.08 | — | — | — | — | — | 0.54 | — | — | — | |
| LossModel=Pythia 6.9B, Attack Category=Loss-Based2025.08 | — | — | — | — | — | 0.55 | — | — | — | |
| LossModel=Pythia 12B, Attack Category=Loss-Based2025.08 | — | — | — | — | — | 0.56 | — | — | — | |
| Min-K%Model=Pythia 1.4B, Attack Category=Loss-Based2025.08 | — | — | — | — | — | — | — | — | 0.54 | |
| Min-K%Model=Pythia 2.8B, Attack Category=Loss-Based2025.08 | — | — | — | — | — | — | — | — | 0.54 | |
| Min-K%Model=Pythia 6.9B, Attack Category=Loss-Based2025.08 | — | — | — | — | — | — | — | — | 0.55 | |
| Min-K%Model=Pythia 12B, Attack Category=Loss-Based2025.08 | — | — | — | — | — | — | — | — | 0.56 | |
| Reference LossModel=Pythia 1.4B, Attack Category=Loss-Based2025.08 | — | — | — | — | — | — | 0.56 | — | — | |
| Reference LossModel=Pythia 2.8B, Attack Category=Loss-Based2025.08 | — | — | — | — | — | — | 0.58 | — | — | |
| Reference LossModel=Pythia 6.9B, Attack Category=Loss-Based2025.08 | — | — | — | — | — | — | 0.6 | — | — | |
| Reference LossModel=Pythia 12B, Attack Category=Loss-Based2025.08 | — | — | — | — | — | — | 0.62 | — | — | |
| zlibModel=Pythia 1.4B, Attack Category=Loss-Based2025.08 | — | — | — | — | — | — | — | 0.53 | — | |
| zlibModel=Pythia 2.8B, Attack Category=Loss-Based2025.08 | — | — | — | — | — | — | — | 0.54 | — | |
| zlibModel=Pythia 6.9B, Attack Category=Loss-Based2025.08 | — | — | — | — | — | — | — | 0.55 | — | |
| zlibModel=Pythia 12B, Attack Category=Loss-Based2025.08 | — | — | — | — | — | — | — | 0.55 | — |