Membership Inference Attack on Wikipedia
0.9AUCLiRA
Evaluation results
| Method | Links | |
|---|---|---|
| LiRAReference Model=Oracle2023.05 | 0.9 | |
| ReferenceModel Parameters=6.9B, Access Setting=Gray-Box2026.01 | 0.678 | |
| ReferenceModel Parameters=2.8B, Access Setting=Gray-Box2026.01 | 0.676 | |
| LossModel Parameters=6.9B, Access Setting=Gray-Box2026.01 | 0.664 | |
| LossModel Parameters=2.8B, Access Setting=Gray-Box2026.01 | 0.662 | |
| Min-K%Model Parameters=6.9B, Access Setting=Gray-Box2026.01 | 0.658 | |
| Min-K%Model Parameters=2.8B, Access Setting=Gray-Box2026.01 | 0.656 | |
| ReCaLLModel Parameters=6.9B, Access Setting=Gray-Box2026.01 | 0.652 | |
| ReCaLLModel Parameters=2.8B, Access Setting=Gray-Box2026.01 | 0.651 | |
| SimMIAModel Parameters=2.8B, Access Setting=Black-Box, Scoring Function=Eq. 32026.01 | 0.65 | |
| LossModel Parameters=1.4B, Access Setting=Gray-Box2026.01 | 0.649 | |
| SimMIAModel Parameters=6.9B, Access Setting=Black-Box, Scoring Function=Eq. 32026.01 | 0.648 | |
| Min-K%++Model Parameters=2.8B, Access Setting=Gray-Box2026.01 | 0.642 | |
| LowercaseModel Parameters=6.9B, Access Setting=Gray-Box2026.01 | 0.641 | |
| SimMIAModel Parameters=1.4B, Access Setting=Black-Box, Scoring Function=Eq. 32026.01 | 0.641 | |
| Min-K%Model Parameters=1.4B, Access Setting=Gray-Box2026.01 | 0.638 | |
| Min-K%++Model Parameters=6.9B, Access Setting=Gray-Box2026.01 | 0.638 | |
| PETALModel Parameters=2.8B, Access Setting=Gray-Box2026.01 | 0.637 | |
| PETALModel Parameters=6.9B, Access Setting=Gray-Box2026.01 | 0.636 | |
| LowercaseModel Parameters=2.8B, Access Setting=Gray-Box2026.01 | 0.635 | |
| ReCaLLModel Parameters=1.4B, Access Setting=Gray-Box2026.01 | 0.634 | |
| SimMIAModel Parameters=6.9B, Access Setting=Black-Box, Scoring Function=Eq. 12026.01 | 0.631 | |
| ZlibModel Parameters=2.8B, Access Setting=Gray-Box2026.01 | 0.63 | |
| ZlibModel Parameters=6.9B, Access Setting=Gray-Box2026.01 | 0.63 | |
| ReferenceModel Parameters=1.4B, Access Setting=Gray-Box2026.01 | 0.629 | |
| PETALModel Parameters=1.4B, Access Setting=Gray-Box2026.01 | 0.624 | |
| LossModel Parameters=160M, Access Setting=Gray-Box2026.01 | 0.623 | |
| LowercaseModel Parameters=1.4B, Access Setting=Gray-Box2026.01 | 0.621 | |
| Min-K%++Model Parameters=1.4B, Access Setting=Gray-Box2026.01 | 0.621 | |
| SimMIAModel Parameters=1.4B, Access Setting=Black-Box, Scoring Function=Eq. 12026.01 | 0.621 | |
| Neighbour Attack2023.05 | 0.62 | |
| NeighborModel Parameters=2.8B, Access Setting=Gray-Box2026.01 | 0.618 | |
| NeighborModel Parameters=6.9B, Access Setting=Gray-Box2026.01 | 0.617 | |
| SimMIAModel Parameters=160M, Access Setting=Black-Box, Scoring Function=Eq. 32026.01 | 0.617 | |
| SimMIAModel Parameters=2.8B, Access Setting=Black-Box, Scoring Function=Eq. 12026.01 | 0.613 | |
| ZlibModel Parameters=1.4B, Access Setting=Gray-Box2026.01 | 0.609 | |
| NeighborModel Parameters=1.4B, Access Setting=Gray-Box2026.01 | 0.609 | |
| PETALModel Parameters=160M, Access Setting=Gray-Box2026.01 | 0.601 | |
| Min-K%Model Parameters=160M, Access Setting=Gray-Box2026.01 | 0.599 | |
| LowercaseModel Parameters=160M, Access Setting=Gray-Box2026.01 | 0.586 | |
| NeighborModel Parameters=160M, Access Setting=Gray-Box2026.01 | 0.579 | |
| SimMIAModel Parameters=160M, Access Setting=Black-Box, Scoring Function=Eq. 12026.01 | 0.573 | |
| ZlibModel Parameters=160M, Access Setting=Gray-Box2026.01 | 0.565 | |
| ReCaLLModel Parameters=160M, Access Setting=Gray-Box2026.01 | 0.559 | |
| Min-K%++Model Parameters=160M, Access Setting=Gray-Box2026.01 | 0.552 | |
| LiRAReference Model=Base2023.05 | 0.54 | |
| Name Cloze QueriesModel=Claude Sonnet 3.52026.04 | 0.538 | |
| Familiarity RankingModel=Llama 3.1 70b2026.04 | 0.531 | |
| Familiarity RankingModel=GPT-4o2026.04 | 0.528 | |
| SaMIAModel Parameters=6.9B, Access Setting=Black-Box2026.01 | 0.521 | |
| Familiarity RankingModel=Claude Sonnet 3.52026.04 | 0.521 | |
| Name Cloze QueriesModel=Llama 3.1 70b2026.04 | 0.521 | |
| LOSS Attack2023.05 | 0.52 | |
| SaMIAModel Parameters=2.8B, Access Setting=Black-Box2026.01 | 0.519 | |
| Familiarity RankingModel=GPT4o-mini2026.04 | 0.519 | |
| Name Cloze QueriesModel=GPT4o-mini2026.04 | 0.519 | |
| ReferenceModel Parameters=160M, Access Setting=Gray-Box2026.01 | 0.517 | |
| DE-COPModel=Mixtral 8x7b2026.04 | 0.516 | |
| ProbingModel=Llama 3.1 70b2026.04 | 0.516 | |
| SaMIAModel Parameters=1.4B, Access Setting=Black-Box2026.01 | 0.513 | |
| Familiarity RankingModel=GPT 3.5 Turbo2026.04 | 0.509 | |
| DE-COPModel=GPT4o-mini2026.04 | 0.502 | |
| ProbingModel=GPT 3.5 Turbo2026.04 | 0.502 | |
| DE-COPModel=Claude Sonnet 3.52026.04 | 0.502 | |
| ProbingModel=Claude Sonnet 3.52026.04 | 0.502 | |
| Name Cloze QueriesModel=GPT-4o2026.04 | 0.5 | |
| ProbingModel=GPT4o-mini2026.04 | 0.5 | |
| Name Cloze QueriesModel=GPT 3.5 Turbo2026.04 | 0.5 | |
| DE-COPModel=GPT-4o2026.04 | 0.498 | |
| ProbingModel=GPT-4o2026.04 | 0.498 | |
| ProbingModel=Mixtral 8x7b2026.04 | 0.498 | |
| Familiarity RankingModel=Mixtral 8x7b2026.04 | 0.493 | |
| SaMIAModel Parameters=160M, Access Setting=Black-Box2026.01 | 0.491 | |
| Name Cloze QueriesModel=Mixtral 8x7b2026.04 | 0.481 | |
| DE-COPModel=GPT 3.5 Turbo2026.04 | 0.46 |