Membership Inference Attack on MedInstruct
97.7AUCICP-MIA-SP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ICP-MIA-SPTarget Model=LLaMA-3.2-3B2025.12 | 97.7 | 53.8 | |
| ICP-MIA-SPTarget Model=LLaMA-3.2-3B-Instruct2025.12 | 95.9 | 32.6 | |
| SPV-MIATarget Model=LLaMA-3.2-3B-Instruct2025.12 | 94.6 | 48.6 | |
| SPV-MIATarget Model=LLaMA-3.2-3B2025.12 | 93.2 | 60.8 | |
| ZlibTarget Model=LLaMA-3.2-3B-Instruct2025.12 | 92.1 | 9.6 | |
| ZlibTarget Model=LLaMA-3.2-3B2025.12 | 91.7 | 7.6 | |
| Min-K%Target Model=LLaMA-3.2-3B2025.12 | 90.8 | 41.2 | |
| RecallTarget Model=LLaMA-3.2-3B2025.12 | 90.8 | 4.4 | |
| Loss AttackTarget Model=LLaMA-3.2-3B-Instruct2025.12 | 90.7 | 26.6 | |
| Min-K%Target Model=LLaMA-3.2-3B-Instruct2025.12 | 90.7 | 28.8 | |
| Loss AttackTarget Model=LLaMA-3.2-3B2025.12 | 90.4 | 25.6 | |
| Reference Attack (Ref)Target Model=LLaMA-3.2-3B-Instruct2025.12 | 90.2 | 16.6 | |
| RecallTarget Model=LLaMA-3.2-3B-Instruct2025.12 | 89.9 | 13.3 | |
| Reference Attack (Ref)Target Model=LLaMA-3.2-3B2025.12 | 89.8 | 14.2 | |
| Reference Attack (Ref)Target Model=Pythia-2.8B-deduped2025.12 | 89.1 | 41.4 | |
| Reference Attack (Base)Target Model=LLaMA-3.2-3B-Instruct2025.12 | 88.5 | 7.8 | |
| ICP-MIA-SPTarget Model=Pythia-2.8B-deduped2025.12 | 88.2 | 27 | |
| Reference Attack (Base)Target Model=LLaMA-3.2-3B2025.12 | 87.8 | 8.2 | |
| Reference Attack (Base)Target Model=Pythia-2.8B-deduped2025.12 | 87.1 | 24.4 | |
| SPV-MIATarget Model=Pythia-2.8B-deduped2025.12 | 86.9 | 37.4 | |
| ZlibTarget Model=Pythia-2.8B-deduped2025.12 | 86.6 | 2.8 | |
| Min-K%Target Model=Pythia-2.8B-deduped2025.12 | 86.5 | 17.6 | |
| RecallTarget Model=Pythia-2.8B-deduped2025.12 | 85.4 | 16.2 | |
| Loss AttackTarget Model=Pythia-2.8B-deduped2025.12 | 84.9 | 16.8 | |
| ICP-MIA-RefTarget Model=LLaMA-3.2-3B-Instruct2025.12 | 83.8 | 4.4 | |
| Min-K%++Target Model=LLaMA-3.2-3B-Instruct2025.12 | 81 | 9 | |
| Min-K%++Target Model=LLaMA-3.2-3B2025.12 | 78.7 | 6 | |
| ICP-MIA-RefTarget Model=LLaMA-3.2-3B2025.12 | 77.5 | 1.8 | |
| Min-K%++Target Model=Pythia-2.8B-deduped2025.12 | 75.8 | 3.6 | |
| ICP-MIA-RefTarget Model=Pythia-2.8B-deduped2025.12 | 74.6 | 7.4 | |
| NeighborhoodTarget Model=LLaMA-3.2-3B-Instruct2025.12 | 55.6 | 0.8 | |
| NeighborhoodTarget Model=LLaMA-3.2-3B2025.12 | 53.5 | 1 | |
| NeighborhoodTarget Model=Pythia-2.8B-deduped2025.12 | 52.7 | 1 | |
| Bag of WordsTarget Model=LLaMA-3.2-3B-Instruct2025.12 | 51.2 | 1.4 | |
| Bag of WordsTarget Model=Pythia-2.8B-deduped2025.12 | 49.7 | 0.2 | |
| Bag of WordsTarget Model=LLaMA-3.2-3B2025.12 | 49.3 | 1 |