Membership Inference Attack on ArXiv
85AUCSAMA
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| SAMA2026.01 | 85 | 58.6 | 17.8 | 1.4 | — | |
| SAMAModel=Dream-v0-Base-7B2026.01 | 74.8 | 41.2 | 9.8 | 1.1 | — | |
| AutoMIATarget model=OPT 7B2026.03 | 70.3 | — | 10 | — | 28.5 | |
| AutoMIATarget model=Pythia 1.4B2026.03 | 68.7 | — | 10.8 | — | 26.9 | |
| Ratio2026.01 | 59.7 | 18.1 | 2.3 | 0.1 | — | |
| OpenEvolveTarget model=OPT 7B2026.03 | 59.7 | — | — | — | 10 | |
| OpenEvolveTarget model=Pythia 1.4B2026.03 | 59.3 | — | 3.6 | — | 9.6 | |
| Familiarity RankingModel=GPT-4o2026.04 | 56.8 | — | — | — | — | |
| Familiarity RankingModel=Claude Sonnet 3.52026.04 | 56.3 | — | — | — | — | |
| Name Cloze QueriesModel=GPT-4o2026.04 | 55.6 | — | — | — | — | |
| Name Cloze QueriesModel=GPT 3.5 Turbo2026.04 | 54.8 | — | — | — | — | |
| DE-COPModel=Claude Sonnet 3.52026.04 | 54.8 | — | — | — | — | |
| Hallinan et al.Target model=Pythia 1.4B2026.03 | 54.7 | — | 6 | — | 10.4 | |
| Name Cloze QueriesModel=Llama 3.1 70b2026.04 | 54.7 | — | — | — | — | |
| Hallinan et al.Target model=OPT 7B2026.03 | 54.2 | — | 2 | — | 6.8 | |
| Min-K%Model=Dream-v0-Base-7B2026.01 | 53.2 | 12.5 | 1.5 | 0.1 | — | |
| DE-COPModel=GPT-4o2026.04 | 52.9 | — | — | — | — | |
| PIA2026.01 | 52.5 | 9.9 | 1.1 | 0 | — | |
| DE-COPModel=Mixtral 8x7b2026.04 | 52.3 | — | — | — | — | |
| RatioModel=Dream-v0-Base-7B2026.01 | 52.1 | 11.8 | 1.6 | 0.1 | — | |
| SecMI2026.01 | 52 | 9.6 | 0.6 | 0.1 | — | |
| BoWsModel=Dream-v0-Base-7B2026.01 | 51.9 | 10.7 | 1.1 | 0 | — | |
| BoWs2026.01 | 51.9 | 10.7 | 1.1 | 0 | — | |
| Name Cloze QueriesModel=Claude Sonnet 3.52026.04 | 51.9 | — | — | — | — | |
| Lowercase2026.01 | 51.5 | 10.3 | 1.3 | 0.1 | — | |
| SecMIModel=Dream-v0-Base-7B2026.01 | 51.3 | 10.8 | 1 | 0 | — | |
| Name Cloze QueriesModel=Mixtral 8x7b2026.04 | 51.2 | — | — | — | — | |
| LowercaseModel=Dream-v0-Base-7B2026.01 | 50.9 | 9.7 | 1 | 0.1 | — | |
| NeighborModel=Dream-v0-Base-7B2026.01 | 50.6 | 10.1 | 1.1 | 0.1 | — | |
| Loss2026.01 | 50.6 | 11.9 | 1 | 0 | — | |
| Neighbor2026.01 | 50.6 | 9.8 | 1.2 | 0.1 | — | |
| ProbingModel=Llama 3.1 70b2026.04 | 50.5 | — | — | — | — | |
| Min-K%++Model=Dream-v0-Base-7B2026.01 | 50.4 | 10.6 | 0.9 | 0 | — | |
| PIAModel=Dream-v0-Base-7B2026.01 | 50.2 | 9.6 | 0.8 | 0 | — | |
| ReCallModel=Dream-v0-Base-7B2026.01 | 50.1 | 10.9 | 0.9 | 0 | — | |
| ReCall2026.01 | 50.1 | 13.2 | 0.7 | 0 | — | |
| CON-Recall2026.01 | 50 | 10.1 | 1.1 | 0 | — | |
| ProbingModel=GPT-4o2026.04 | 50 | — | — | — | — | |
| ProbingModel=GPT4o-mini2026.04 | 50 | — | — | — | — | |
| ProbingModel=GPT 3.5 Turbo2026.04 | 50 | — | — | — | — | |
| ProbingModel=Claude Sonnet 3.52026.04 | 50 | — | — | — | — | |
| ProbingModel=Mixtral 8x7b2026.04 | 50 | — | — | — | — | |
| LossModel=Dream-v0-Base-7B2026.01 | 49.8 | 10.3 | 0.9 | 0 | — | |
| Name Cloze QueriesModel=GPT4o-mini2026.04 | 49.6 | — | — | — | — | |
| Familiarity RankingModel=Llama 3.1 70b2026.04 | 49.6 | — | — | — | — | |
| CON-RecallModel=Dream-v0-Base-7B2026.01 | 49.4 | 9.8 | 0.8 | 0 | — | |
| Familiarity RankingModel=GPT4o-mini2026.04 | 49.1 | — | — | — | — | |
| ZLIB2026.01 | 49 | 11.9 | 1.2 | 0 | — | |
| Min-K%2026.01 | 48.8 | 10.2 | 1.2 | 0.1 | — | |
| Familiarity RankingModel=GPT 3.5 Turbo2026.04 | 48.7 | — | — | — | — | |
| ZLIBModel=Dream-v0-Base-7B2026.01 | 48.5 | 9.1 | 0.7 | 0 | — | |
| Min-K%++2026.01 | 48.5 | 9.5 | 0.6 | 0 | — | |
| DE-COPModel=GPT 3.5 Turbo2026.04 | 48.2 | — | — | — | — | |
| Familiarity RankingModel=Mixtral 8x7b2026.04 | 47.2 | — | — | — | — | |
| DE-COPModel=GPT4o-mini2026.04 | 41.9 | — | — | — | — |