Membership Inference Attack on VL-MIA Flickr-10k
0.76AUCCSA-MIA
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CSA-MIATarget Model=LLaVA-1.5, Type=Black-Box2026.05 | 0.76 | 0.163 | 21.3 | 81 | 58.2 | |
| CSA-MIATarget Model=LLaMA Adapter v2, Type=Black-Box2026.05 | 0.751 | 0.159 | 20.9 | 80.7 | 57.9 | |
| CSA-MIATarget Model=MiniGPT-4, Type=Black-Box2026.05 | 0.712 | 0.102 | 15.2 | 75.2 | 55.1 | |
| MaxRényi_100%Target Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.709 | 0.166 | 21.6 | 81.2 | 58.3 | |
| MaxRényi_100%Target Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.707 | 0.113 | 16.3 | 76.6 | 55.7 | |
| MaxRényi_100%Target Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.707 | 0.138 | 18.8 | 79 | 56.9 | |
| MaxRényi_10%Target Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.703 | 0.14 | 19 | 79.1 | 57 | |
| ModRényiTarget Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.703 | 0.144 | 19.4 | 79.5 | 57.2 | |
| PerplexityTarget Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.695 | 0.107 | 15.7 | 75.9 | 55.4 | |
| Min-20% ++Target Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.682 | 0.088 | 13.8 | 73.5 | 54.4 | |
| MaxRényi_10%Target Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.679 | 0.069 | 11.8 | 70.5 | 53.4 | |
| Max_Prob_GapTarget Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.676 | 0.135 | 18.5 | 78.7 | 56.8 | |
| Min-20% ++Target Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.67 | 0.084 | 13.4 | 72.9 | 54.2 | |
| PerplexityTarget Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.663 | 0.099 | 14.9 | 74.9 | 54.9 | |
| Min-20% ++Target Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.663 | 0.113 | 16.3 | 76.5 | 55.6 | |
| MaxRényi_0%Target Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.661 | 0.057 | 10.7 | 68.2 | 52.9 | |
| MaxRényi_10%Target Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.658 | 0.098 | 14.8 | 74.8 | 54.9 | |
| ModRényiTarget Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.654 | 0.089 | 13.9 | 73.7 | 54.5 | |
| Max_Prob_GapTarget Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.65 | 0.093 | 14.3 | 74 | 54.6 | |
| Min_20% ProbTarget Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.649 | 0.059 | 10.8 | 68.8 | 53 | |
| Min-10% ++Target Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.648 | 0.076 | 12.6 | 71.6 | 53.8 | |
| Max_Prob_GapTarget Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.644 | 0.11 | 16 | 76.2 | 55.5 | |
| Min_20% ProbTarget Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.637 | 0.062 | 11.2 | 69.1 | 53.1 | |
| Min-0% ++Target Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.634 | 0.04 | 9 | 64.2 | 52 | |
| Min-10% ++Target Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.634 | 0.04 | 9 | 64.2 | 52 | |
| Min_10% ProbTarget Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.633 | 0.047 | 9.6 | 66 | 52.3 | |
| MaxRényi_0%Target Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.632 | 0.106 | 15.6 | 75.8 | 55.3 | |
| Min-10% ++Target Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.626 | 0.041 | 9.1 | 64.5 | 52.1 | |
| Min_10% ProbTarget Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.624 | 0.049 | 9.9 | 66.5 | 52.5 | |
| PerplexityTarget Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.614 | 0.045 | 9.5 | 65.6 | 52.3 | |
| Min-0% ++Target Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.608 | 0.052 | 10.2 | 67.1 | 52.6 | |
| ModRényiTarget Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.602 | 0.04 | 9 | 64.3 | 52 | |
| Min_0% ProbTarget Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.601 | 0.029 | 7.9 | 61.2 | 51.5 | |
| Aug_KLTarget Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.599 | 0.072 | 12.2 | 71.1 | 53.6 | |
| Min_0% ProbTarget Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.59 | 0.035 | 8.5 | 63.1 | 51.8 | |
| Image-only InferenceTarget Model=LLaVA-1.5, Type=Black-Box2026.05 | 0.578 | 0.011 | 6.1 | 54.8 | 50.5 | |
| Min_10% ProbTarget Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.565 | 0.024 | 7.4 | 59.7 | 51.2 | |
| Min_20% ProbTarget Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.548 | 0.035 | 8.5 | 63 | 51.8 | |
| Min-0% ++Target Model=LLaVA-1.5, Type=Gray-Box2026.05 | 0.548 | -0.012 | 3.8 | 43.4 | 49.4 | |
| MaxRényi_0%Target Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.546 | 0.002 | 5.2 | 51.2 | 50.1 | |
| Min_0% ProbTarget Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.536 | 0.015 | 6.5 | 56.5 | 50.8 | |
| Aug_KLTarget Model=MiniGPT-4, Type=Gray-Box2026.05 | 0.506 | 0.018 | 6.8 | 57.8 | 50.9 | |
| Image-only InferenceTarget Model=MiniGPT-4, Type=Black-Box2026.05 | 0.486 | -0.023 | 2.7 | 47.9 | 49.9 | |
| Aug_KLTarget Model=LLaMA Adapter v2, Type=Gray-Box2026.05 | 0.448 | -0.016 | 3.4 | 40.2 | 49.2 | |
| Image-only InferenceTarget Model=LLaMA Adapter v2, Type=Black-Box2026.05 | 0.388 | -0.025 | 2.5 | 33.4 | 48.8 |