Membership Inference Attack on XSum (test)
0.945AUCSPV-MIA
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SPV-MIABackbone=LLaMA-3.2-3B-Instruct2025.12 | 0.945 | — | — | — | |
| ICP-MIA-SPBackbone=LLaMA-3.2-3B, K=202025.12 | 0.941 | — | — | — | |
| ICP-MIA-SPBackbone=LLaMA-3.2-3B-Instruct, K=202025.12 | 0.934 | — | — | — | |
| SPV-MIABackbone=LLaMA-3.2-3B2025.12 | 0.92 | — | — | — | |
| SPV-MIABackbone=Pythia-2.8B-deduped2025.12 | 0.892 | — | — | — | |
| ICP-MIA-SPBackbone=Pythia-2.8B-deduped, K=202025.12 | 0.885 | — | — | — | |
| RecallBackbone=LLaMA-3.2-3B, shots=72025.12 | 0.884 | — | — | — | |
| ReferenceBackbone=LLaMA-3.2-3B-Instruct2025.12 | 0.883 | — | — | — | |
| ReferenceBackbone=LLaMA-3.2-3B2025.12 | 0.865 | — | — | — | |
| RecallBackbone=Pythia-2.8B-deduped, shots=72025.12 | 0.859 | — | — | — | |
| RecallBackbone=LLaMA-3.2-3B-Instruct, shots=72025.12 | 0.849 | — | — | — | |
| ICP-MIA-RefBackbone=LLaMA-3.2-3B-Instruct, K=202025.12 | 0.839 | — | — | — | |
| ReferenceBackbone=Pythia-2.8B-deduped2025.12 | 0.831 | — | — | — | |
| ICP-MIA-RefBackbone=LLaMA-3.2-3B, K=202025.12 | 0.802 | — | — | — | |
| Loss AttackBackbone=LLaMA-3.2-3B2025.12 | 0.8 | — | — | — | |
| Min-K%Backbone=LLaMA-3.2-3B, k=202025.12 | 0.799 | — | — | — | |
| ZlibBackbone=LLaMA-3.2-3B2025.12 | 0.788 | — | — | — | |
| Loss AttackBackbone=Pythia-2.8B-deduped2025.12 | 0.78 | — | — | — | |
| ICP-MIA-RefBackbone=Pythia-2.8B-deduped, K=202025.12 | 0.774 | — | — | — | |
| ZlibBackbone=Pythia-2.8B-deduped2025.12 | 0.767 | — | — | — | |
| Min-K%Backbone=Pythia-2.8B-deduped, k=202025.12 | 0.767 | — | — | — | |
| Loss AttackBackbone=LLaMA-3.2-3B-Instruct2025.12 | 0.765 | — | — | — | |
| ZlibBackbone=LLaMA-3.2-3B-Instruct2025.12 | 0.755 | — | — | — | |
| Min-K%Backbone=LLaMA-3.2-3B-Instruct, k=202025.12 | 0.744 | — | — | — | |
| Min-K%++Backbone=LLaMA-3.2-3B, k=202025.12 | 0.73 | — | — | — | |
| Min-K%++Backbone=Pythia-2.8B-deduped, k=202025.12 | 0.703 | — | — | — | |
| Min-K%++Backbone=LLaMA-3.2-3B-Instruct, k=202025.12 | 0.694 | — | — | — | |
| SAMATarget Model=LLaDA-8B-Base2026.01 | 0.682 | 27.7 | 5.5 | 0.8 | |
| NeighborhoodBackbone=Pythia-2.8B-deduped, neighbors=20, top_k=52025.12 | 0.617 | — | — | — | |
| RatioTarget Model=LLaDA-8B-Base2026.01 | 0.616 | 21.2 | 3.6 | 0.5 | |
| NeighborhoodBackbone=LLaMA-3.2-3B-Instruct, neighbors=20, top_k=52025.12 | 0.612 | — | — | — | |
| NeighborhoodBackbone=LLaMA-3.2-3B, neighbors=20, top_k=52025.12 | 0.604 | — | — | — | |
| LowercaseTarget Model=LLaDA-8B-Base2026.01 | 0.548 | 15.2 | 2 | 0.3 | |
| SecMITarget Model=LLaDA-8B-Base2026.01 | 0.542 | 14.7 | 1.8 | 0.2 | |
| ZLIBTarget Model=LLaDA-8B-Base2026.01 | 0.54 | 14.5 | 1.8 | 0.3 | |
| PIATarget Model=LLaDA-8B-Base2026.01 | 0.538 | 14.4 | 1.7 | 0.2 | |
| Min-K%Target Model=LLaDA-8B-Base2026.01 | 0.536 | 14.1 | 1.9 | 0.2 | |
| ReCallTarget Model=LLaDA-8B-Base2026.01 | 0.535 | 14 | 1.7 | 0.2 | |
| Min-K%++Target Model=LLaDA-8B-Base2026.01 | 0.533 | 13.9 | 1.7 | 0.2 | |
| LossTarget Model=LLaDA-8B-Base2026.01 | 0.532 | 13.8 | 1.7 | 0.2 | |
| CON-RecallTarget Model=LLaDA-8B-Base2026.01 | 0.531 | 13.7 | 1.6 | 0.2 | |
| NeighborTarget Model=LLaDA-8B-Base2026.01 | 0.522 | 12.9 | 1.5 | 0.1 | |
| BoWsTarget Model=LLaDA-8B-Base2026.01 | 0.515 | 12.2 | 1.3 | 0.1 |