Membership Inference Attack on VL-MIA Flickr 2k
0.814AUCCSA-MIA
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CSA-MIATarget Model=LLaVA-1.5, Type=Black-Box2026.05 | 0.814 | 0.342 | 39.2 | 88.7 | 67.1 | |
| CSA-MIATarget Model=LLaMA Adapter v2, Type=Black-Box2026.05 | 0.755 | 0.18 | 23 | 82.1 | 59 | |
| CSA-MIATarget Model=MiniGPT-4, Type=Black-Box2026.05 | 0.722 | 0.092 | 14.2 | 74 | 54.6 | |
| MaxRényi_10%Target Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.716 | 0.134 | 18.4 | 78.6 | 56.7 | |
| MaxRényi_100%Target Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.713 | 0.153 | 20.3 | 80.9 | 57.8 | |
| MaxRényi_100%Target Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.712 | 0.162 | 21.2 | 80.9 | 58.1 | |
| ModRényiTarget Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.688 | 0.127 | 17.7 | 78 | 56.4 | |
| PerplexityTarget Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.683 | 0.089 | 13.9 | 73.9 | 54.5 | |
| MaxRényi_100%Target Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.679 | 0.076 | 12.6 | 72.4 | 53.9 | |
| Min-20% ++Target Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.679 | 0.101 | 15 | 75.4 | 55.1 | |
| MaxRényi_10%Target Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.672 | 0.052 | 10.2 | 67.1 | 52.6 | |
| Max_Prob_GapTarget Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.668 | 0.139 | 18.9 | 79.4 | 57 | |
| MaxRényi_10%Target Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.667 | 0.076 | 12.6 | 71.6 | 53.8 | |
| Min-20% ++Target Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.666 | 0.141 | 19.1 | 79.3 | 57.1 | |
| Min-10% ++Target Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.655 | 0.086 | 13.6 | 73.1 | 54.3 | |
| MaxRényi_0%Target Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.651 | 0.115 | 16.5 | 76.7 | 55.8 | |
| Max_Prob_GapTarget Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.65 | 0.125 | 17.5 | 77.8 | 56.3 | |
| Min-10% ++Target Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.65 | 0.078 | 12.8 | 72.3 | 54 | |
| PerplexityTarget Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.649 | 0.068 | 11.7 | 70.5 | 53.4 | |
| Min-20% ++Target Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.649 | 0.021 | 7.1 | 58.7 | 51.1 | |
| Max_Prob_GapTarget Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.641 | 0.096 | 14.6 | 74.9 | 54.9 | |
| ModRényiTarget Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.641 | 0.071 | 12.1 | 70.8 | 53.6 | |
| Min_20% ProbTarget Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.629 | 0.053 | 10.3 | 67.3 | 52.7 | |
| Min-0% ++Target Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.621 | 0.035 | 8.5 | 63.4 | 51.8 | |
| Min-10% ++Target Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.621 | 0.035 | 8.5 | 63.4 | 51.8 | |
| Min_20% ProbTarget Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.619 | 0.029 | 7.8 | 61.4 | 51.5 | |
| Min_10% ProbTarget Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.614 | 0.033 | 8.3 | 62.4 | 51.7 | |
| MaxRényi_0%Target Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.612 | 0.071 | 12.1 | 70.8 | 53.6 | |
| Min-0% ++Target Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.612 | 0.061 | 11 | 69.2 | 53.1 | |
| PerplexityTarget Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.609 | 0.032 | 8.2 | 62.1 | 51.6 | |
| Min_10% ProbTarget Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.609 | 0.033 | 8.3 | 62.4 | 51.7 | |
| ModRényiTarget Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.602 | 0.032 | 8.2 | 62.1 | 51.6 | |
| Min_20% ProbTarget Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.592 | 0.025 | 7.5 | 60 | 51.3 | |
| Min-0% ++Target Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.588 | -0.012 | 3.8 | 43.2 | 49.4 | |
| Min_0% ProbTarget Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.584 | 0.018 | 6.8 | 57.6 | 50.9 | |
| Min_0% ProbTarget Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.584 | -0.001 | 4.9 | 49.5 | 50 | |
| Min_10% ProbTarget Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.579 | -0.002 | 4.4 | 48.9 | 49.9 | |
| Aug_KLTarget Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.577 | 0.066 | 11.6 | 70.7 | 53.4 | |
| MaxRényi_0%Target Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.562 | -0.014 | 3.5 | 41.7 | 49.3 | |
| Image-only InferenceTarget Model=MiniGPT-4, Type=Black-Box2026.05 | 0.557 | 0.028 | 7.8 | 61.4 | 51.5 | |
| Image-only InferenceTarget Model=LLaVA-1.5, Type=Black-Box2026.05 | 0.556 | 0.017 | 6.7 | 57.3 | 50.9 | |
| Min_0% ProbTarget Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.551 | 0.005 | 5.5 | 52.4 | 50.3 | |
| Aug_KLTarget Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.534 | 0.004 | 4.5 | 49.5 | 50 | |
| Aug_KLTarget Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.469 | -0.018 | 3.2 | 39 | 49.1 | |
| Image-only InferenceTarget Model=LLaMA Adapter v2, Type=Black-Box2026.05 | 0.386 | -0.039 | 1.1 | 18.3 | 48.1 |