Membership Inference Attack on AG News (test)
0.909AUCICP-MIA-SP
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ICP-MIA-SPBackbone=LLaMA-3.2-3B, K=202025.12 | 0.909 | — | — | — | |
| ICP-MIA-SPBackbone=LLaMA-3.2-3B-Instruct, K=202025.12 | 0.905 | — | — | — | |
| SPV-MIABackbone=LLaMA-3.2-3B-Instruct2025.12 | 0.903 | — | — | — | |
| SPV-MIABackbone=LLaMA-3.2-3B2025.12 | 0.901 | — | — | — | |
| ICP-MIA-SPBackbone=Pythia-2.8B-deduped, K=202025.12 | 0.879 | — | — | — | |
| SPV-MIABackbone=Pythia-2.8B-deduped2025.12 | 0.87 | — | — | — | |
| ReferenceBackbone=LLaMA-3.2-3B2025.12 | 0.844 | — | — | — | |
| ReferenceBackbone=LLaMA-3.2-3B-Instruct2025.12 | 0.832 | — | — | — | |
| ICP-MIA-RefBackbone=LLaMA-3.2-3B-Instruct, K=202025.12 | 0.797 | — | — | — | |
| ReferenceBackbone=Pythia-2.8B-deduped2025.12 | 0.786 | — | — | — | |
| ICP-MIA-RefBackbone=LLaMA-3.2-3B, K=202025.12 | 0.767 | — | — | — | |
| ICP-MIA-RefBackbone=Pythia-2.8B-deduped, K=202025.12 | 0.753 | — | — | — | |
| RecallBackbone=LLaMA-3.2-3B-Instruct, shots=72025.12 | 0.743 | — | — | — | |
| RecallBackbone=LLaMA-3.2-3B, shots=72025.12 | 0.733 | — | — | — | |
| RecallBackbone=Pythia-2.8B-deduped, shots=72025.12 | 0.713 | — | — | — | |
| SAMATarget Model=LLaDA-8B-Base2026.01 | 0.673 | 0.268 | 0.052 | 0.007 | |
| ZlibBackbone=Pythia-2.8B-deduped2025.12 | 0.661 | — | — | — | |
| Min-K%Backbone=Pythia-2.8B-deduped, k=202025.12 | 0.659 | — | — | — | |
| Loss AttackBackbone=Pythia-2.8B-deduped2025.12 | 0.658 | — | — | — | |
| Min-K%Backbone=LLaMA-3.2-3B, k=202025.12 | 0.617 | — | — | — | |
| Loss AttackBackbone=LLaMA-3.2-3B2025.12 | 0.612 | — | — | — | |
| RatioTarget Model=LLaDA-8B-Base2026.01 | 0.608 | 0.205 | 0.034 | 0.004 | |
| Min-K%++Backbone=Pythia-2.8B-deduped, k=202025.12 | 0.604 | — | — | — | |
| Min-K%Backbone=LLaMA-3.2-3B-Instruct, k=202025.12 | 0.603 | — | — | — | |
| ZlibBackbone=LLaMA-3.2-3B2025.12 | 0.603 | — | — | — | |
| NeighborhoodBackbone=Pythia-2.8B-deduped, neighbors=20, top_k=52025.12 | 0.601 | — | — | — | |
| Loss AttackBackbone=LLaMA-3.2-3B-Instruct2025.12 | 0.594 | — | — | — | |
| ZlibBackbone=LLaMA-3.2-3B-Instruct2025.12 | 0.586 | — | — | — | |
| NeighborhoodBackbone=LLaMA-3.2-3B, neighbors=20, top_k=52025.12 | 0.581 | — | — | — | |
| NeighborhoodBackbone=LLaMA-3.2-3B-Instruct, neighbors=20, top_k=52025.12 | 0.58 | — | — | — | |
| Min-K%++Backbone=LLaMA-3.2-3B, k=202025.12 | 0.574 | — | — | — | |
| Min-K%++Backbone=LLaMA-3.2-3B-Instruct, k=202025.12 | 0.57 | — | — | — | |
| LowercaseTarget Model=LLaDA-8B-Base2026.01 | 0.542 | 0.148 | 0.019 | 0.003 | |
| SecMITarget Model=LLaDA-8B-Base2026.01 | 0.536 | 0.143 | 0.017 | 0.002 | |
| ZLIBTarget Model=LLaDA-8B-Base2026.01 | 0.534 | 0.141 | 0.018 | 0.002 | |
| PIATarget Model=LLaDA-8B-Base2026.01 | 0.532 | 0.14 | 0.016 | 0.002 | |
| Min-K%Target Model=LLaDA-8B-Base2026.01 | 0.53 | 0.138 | 0.017 | 0.002 | |
| ReCallTarget Model=LLaDA-8B-Base2026.01 | 0.529 | 0.137 | 0.016 | 0.002 | |
| Min-K%++Target Model=LLaDA-8B-Base2026.01 | 0.527 | 0.136 | 0.016 | 0.002 | |
| LossTarget Model=LLaDA-8B-Base2026.01 | 0.526 | 0.135 | 0.015 | 0.002 | |
| CON-RecallTarget Model=LLaDA-8B-Base2026.01 | 0.525 | 0.134 | 0.015 | 0.002 | |
| NeighborTarget Model=LLaDA-8B-Base2026.01 | 0.516 | 0.126 | 0.014 | 0.001 | |
| BoWsTarget Model=LLaDA-8B-Base2026.01 | 0.508 | 0.118 | 0.012 | 0.001 |