Membership Inference Attack on WikiMIA-25
0.961AUCReCaLL
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| ReCaLLAccess Setting=Grey-Box, Target Model=Pythia-6.9B2026.01 | 0.961 | 78.8 | |
| ReCaLLAccess Setting=Grey-Box, Target Model=Qwen3-8B-Base2026.01 | 0.91 | 43.4 | |
| SimMIA*Access Setting=Black-Box, Target Model=Pythia-6.9B, Scoring Method=Eq. 12026.01 | 0.906 | 30.1 | |
| SimMIA*Access Setting=Black-Box, Target Model=GPT-5-Chat, Scoring Method=Eq. 12026.01 | 0.902 | 71.7 | |
| SimMIAAccess Setting=Black-Box, Target Model=Claude-4.5-Haiku, Scoring Method=Eq. 32026.01 | 0.867 | 51.3 | |
| SimMIAAccess Setting=Black-Box, Target Model=Pythia-6.9B, Scoring Method=Eq. 32026.01 | 0.85 | 8.8 | |
| SimMIA*Access Setting=Black-Box, Target Model=Claude-4.5-Haiku, Scoring Method=Eq. 12026.01 | 0.808 | 37.2 | |
| PETALAccess Setting=Grey-Box, Target Model=GPT-5-Chat2026.01 | 0.78 | 35.4 | |
| SimMIA*Access Setting=Black-Box, Target Model=Gemini-2.5-Flash, Scoring Method=Eq. 12026.01 | 0.774 | 32.7 | |
| Min-K%Access Setting=Grey-Box, Target Model=Pythia-6.9B2026.01 | 0.773 | 38.9 | |
| SimMIAAccess Setting=Black-Box, Target Model=GPT-5-Chat, Scoring Method=Eq. 32026.01 | 0.757 | 21.2 | |
| SimMIAAccess Setting=Black-Box, Target Model=Gemini-2.5-Flash, Scoring Method=Eq. 32026.01 | 0.754 | 29.2 | |
| SimMIAAccess Setting=Black-Box, Target Model=Qwen3-8B-Base, Scoring Method=Eq. 32026.01 | 0.749 | 4.4 | |
| LossAccess Setting=Grey-Box, Target Model=Pythia-6.9B2026.01 | 0.73 | 36.3 | |
| Min-K%++Access Setting=Grey-Box, Target Model=Pythia-6.9B2026.01 | 0.727 | 41.6 | |
| ZlibAccess Setting=Grey-Box, Target Model=Pythia-6.9B2026.01 | 0.722 | 31.9 | |
| PETALAccess Setting=Grey-Box, Target Model=Pythia-6.9B2026.01 | 0.722 | 30.1 | |
| LowercaseAccess Setting=Grey-Box, Target Model=Pythia-6.9B2026.01 | 0.707 | 23 | |
| PETALAccess Setting=Grey-Box, Target Model=Gemini-2.5-Flash2026.01 | 0.671 | 23.9 | |
| SaMIAAccess Setting=Black-Box, Target Model=GPT-5-Chat2026.01 | 0.657 | 13.3 | |
| ReferenceAccess Setting=Grey-Box, Target Model=Pythia-6.9B2026.01 | 0.652 | 8.9 | |
| SimMIA*Access Setting=Black-Box, Target Model=Qwen3-8B-Base, Scoring Method=Eq. 12026.01 | 0.629 | 5.3 | |
| PETALAccess Setting=Grey-Box, Target Model=Qwen3-8B-Base2026.01 | 0.628 | 18.6 | |
| Min-K%++Access Setting=Grey-Box, Target Model=Qwen3-8B-Base2026.01 | 0.627 | 20.4 | |
| SaMIAAccess Setting=Black-Box, Target Model=Claude-4.5-Haiku2026.01 | 0.626 | 8 | |
| LowercaseAccess Setting=Grey-Box, Target Model=Qwen3-8B-Base2026.01 | 0.594 | 19.5 | |
| LossAccess Setting=Grey-Box, Target Model=Qwen3-8B-Base2026.01 | 0.591 | 11.5 | |
| SaMIAAccess Setting=Black-Box, Target Model=Pythia-6.9B2026.01 | 0.578 | 8.8 | |
| ZlibAccess Setting=Grey-Box, Target Model=Qwen3-8B-Base2026.01 | 0.575 | 8.9 | |
| SaMIAAccess Setting=Black-Box, Target Model=Gemini-2.5-Flash2026.01 | 0.57 | 15 | |
| Min-K%Access Setting=Grey-Box, Target Model=Qwen3-8B-Base2026.01 | 0.568 | 14.2 | |
| ReferenceAccess Setting=Grey-Box, Target Model=Qwen3-8B-Base2026.01 | 0.539 | 3.5 | |
| SaMIAAccess Setting=Black-Box, Target Model=Qwen3-8B-Base2026.01 | 0.503 | 2.7 |