Membership Inference Attack on VL-MIA Flickr v1 (test)
0.812AUCCSA-MIA
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CSA-MIAType=Black-Box, Model=LLaMA Adapter v2, FPR Threshold=1%2026.05 | 0.812 | 0.093 | 10.3 | 91.2 | 54.7 | |
| CSA-MIAType=Black-Box, Model=LLaMA Adapter v2, FPR Threshold=5%2026.05 | 0.812 | 0.207 | 25.7 | 83.7 | 60.3 | |
| CSA-MIAType=Black-Box, Model=LLaMA Adapter v2, FPR Threshold=10%2026.05 | 0.812 | 0.333 | 43.3 | 81.3 | 66.7 | |
| MaxRényi_Max_100%Type=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=1%2026.05 | 0.685 | 0.03 | 4 | 85.7 | 51.7 | |
| MaxRényi_Max_100%Type=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=5%2026.05 | 0.685 | 0.197 | 24.7 | 83.2 | 59.8 | |
| MaxRényi_Max_100%Type=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=10%2026.05 | 0.685 | 0.257 | 35 | 79 | 62.8 | |
| PerplexityType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=1%2026.05 | 0.645 | 0.043 | 5.3 | 84.2 | 52.2 | |
| PerplexityType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=5%2026.05 | 0.645 | 0.077 | 12.7 | 71.7 | 53.8 | |
| PerplexityType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=10%2026.05 | 0.645 | 0.17 | 27 | 73 | 58.5 | |
| MaxRényi_Max_10%Type=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=1%2026.05 | 0.641 | 0.023 | 3.3 | 76.9 | 51.2 | |
| MaxRényi_Max_10%Type=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=5%2026.05 | 0.641 | 0.047 | 9.7 | 67.4 | 52.5 | |
| MaxRényi_Max_10%Type=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=10%2026.05 | 0.641 | 0.093 | 19.3 | 65.9 | 54.7 | |
| ModRényiType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=1%2026.05 | 0.633 | 0.047 | 5.3 | 88.9 | 52.3 | |
| ModRényiType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=5%2026.05 | 0.633 | 0.08 | 12.7 | 73.1 | 54 | |
| ModRényiType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=10%2026.05 | 0.633 | 0.12 | 22 | 68.8 | 56 | |
| Min_20% ProbType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=1%2026.05 | 0.63 | 0.013 | 2.3 | 70 | 50.7 | |
| Min_20% ProbType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=5%2026.05 | 0.63 | 0.07 | 12 | 70.6 | 53.5 | |
| Min_20% ProbType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=10%2026.05 | 0.63 | 0.083 | 18.3 | 64.7 | 54.2 | |
| Max_Prob_GapType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=1%2026.05 | 0.627 | 0.033 | 4.3 | 81.3 | 51.7 | |
| Max_Prob_GapType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=5%2026.05 | 0.627 | 0.127 | 17.3 | 78.8 | 56.3 | |
| Max_Prob_GapType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=10%2026.05 | 0.627 | 0.16 | 26 | 72.2 | 58 | |
| Min_10% ProbType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=1%2026.05 | 0.622 | -0.003 | 0.7 | 40 | 49.8 | |
| Min_10% ProbType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=5%2026.05 | 0.622 | 0.047 | 9.7 | 65.9 | 52.3 | |
| Min_10% ProbType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=10%2026.05 | 0.622 | 0.057 | 15.7 | 61 | 52.8 | |
| Min-20% ++Type=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=1%2026.05 | 0.622 | 0.007 | 1.7 | 62.5 | 50.3 | |
| Min-20% ++Type=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=5%2026.05 | 0.622 | 0.12 | 17 | 77.3 | 56 | |
| Min-20% ++Type=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=10%2026.05 | 0.622 | 0.14 | 24 | 70.6 | 57 | |
| Min_0% ProbType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=1%2026.05 | 0.612 | -0.007 | 0.3 | 25 | 49.7 | |
| Min_0% ProbType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=5%2026.05 | 0.612 | -0.003 | 4.7 | 48.3 | 49.8 | |
| Min_0% ProbType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=10%2026.05 | 0.612 | 0.09 | 19 | 65.5 | 54.5 | |
| Min-10% ++Type=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=1%2026.05 | 0.607 | 0.047 | 5.7 | 85 | 52.3 | |
| Min-10% ++Type=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=5%2026.05 | 0.607 | 0.07 | 12 | 70.6 | 53.5 | |
| Min-10% ++Type=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=10%2026.05 | 0.607 | 0.123 | 22 | 69.5 | 56.2 | |
| MaxRényi_Max_0%Type=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=1%2026.05 | 0.597 | 0 | 0.3 | 50 | 50 | |
| MaxRényi_Max_0%Type=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=5%2026.05 | 0.597 | -0.003 | 4 | 48 | 49.8 | |
| MaxRényi_Max_0%Type=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=10%2026.05 | 0.597 | 0.073 | 17.3 | 63.4 | 53.7 | |
| Min-0% ++Type=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=1%2026.05 | 0.537 | 0.003 | 0.3 | 100 | 50.2 | |
| Min-0% ++Type=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=5%2026.05 | 0.537 | 0.03 | 7 | 63.6 | 51.5 | |
| Min-0% ++Type=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=10%2026.05 | 0.537 | -0.007 | 9 | 48.2 | 49.7 | |
| Image-only InferenceType=Black-Box, Model=LLaMA Adapter v2, FPR Threshold=1%2026.05 | 0.51 | -0.01 | 0 | 0 | 49.5 | |
| Image-only InferenceType=Black-Box, Model=LLaMA Adapter v2, FPR Threshold=5%2026.05 | 0.51 | -0.003 | 4.3 | 48.1 | 49.8 | |
| Image-only InferenceType=Black-Box, Model=LLaMA Adapter v2, FPR Threshold=10%2026.05 | 0.51 | 0 | 10 | 50 | 50 | |
| Aug_KLType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=1%2026.05 | 0.491 | -0.003 | 0 | 0 | 49.8 | |
| Aug_KLType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=5%2026.05 | 0.491 | -0.01 | 3.3 | 43.5 | 49.5 | |
| Aug_KLType=Gray-Box, Model=LLaMA Adapter v2, FPR Threshold=10%2026.05 | 0.491 | -0.007 | 9.3 | 48.3 | 49.7 |