Membership Inference Attack on Flickr
27.7TPR @ 5% FPRAutoMIA
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| AutoMIALogit Type=Text Logits2026.03 | 27.7 | 71.9 | 14.5 | |
| Max Prob GapTarget Model=LLAMA Adapter, Input Slice=desp2026.04 | 23 | — | — | |
| RényiTarget Model=LLaVA, Input Slice=inst, alpha=0.5, type=MAX-0%2026.04 | 21.7 | — | — | |
| RényiTarget Model=LLaVA, Input Slice=inst+desp, alpha=0.5, type=MAX-0%2026.04 | 21.3 | — | — | |
| AutoMIATarget Model=LLaVA, Input Slice=inst+desp, reasoner=DEEPSEEK-V3.2-REASONER2026.04 | 21 | — | — | |
| OpenEvolveLogit Type=Text Logits2026.03 | 20.8 | 63.6 | 5.7 | |
| AutoMIALogit Type=Image Logits2026.03 | 18.9 | 77 | 4.4 | |
| Max Prob GapTarget Model=LLAMA Adapter, Input Slice=inst+desp2026.04 | 18.3 | — | — | |
| RényiTarget Model=LLAMA Adapter, Input Slice=inst, alpha=1, type=MAX-10%2026.04 | 18 | — | — | |
| AutoMIATarget Model=LLAMA Adapter, Input Slice=inst, reasoner=DEEPSEEK-V3.2-REASONER2026.04 | 17.7 | — | — | |
| Li et al., 2024Logit Type=Text Logits2026.03 | 16.4 | 73.6 | 7.5 | |
| Max Prob GapTarget Model=LLaVA, Input Slice=desp2026.04 | 16 | — | — | |
| Max Prob GapTarget Model=LLaVA, Input Slice=inst+desp2026.04 | 16 | — | — | |
| RényiTarget Model=LLaVA, Input Slice=inst, alpha=1, type=MAX-10%2026.04 | 15.3 | — | — | |
| AutoMIATarget Model=LLaVA, Input Slice=inst, reasoner=DEEPSEEK-V3.2-REASONER2026.04 | 14.3 | — | — | |
| AutoMIATarget Model=LLaVA, Input Slice=desp, reasoner=DEEPSEEK-V3.2-REASONER2026.04 | 14.3 | — | — | |
| Li et al., 2024Logit Type=Image Logits2026.03 | 13.2 | 59 | 3.1 | |
| PerplexityTarget Model=LLaVA, Input Slice=desp2026.04 | 13 | — | — | |
| Min-k ProbTarget Model=LLaVA, Input Slice=desp, k_threshold=20%2026.04 | 13 | — | — | |
| AutoMIATarget Model=MiniGPT-4, Input Slice=inst+desp, reasoner=DEEPSEEK-V3.2-REASONER2026.04 | 12.3 | — | — | |
| AutoMIATarget Model=LLAMA Adapter, Input Slice=inst+desp, reasoner=DEEPSEEK-V3.2-REASONER2026.04 | 12.3 | — | — | |
| RényiTarget Model=LLaVA, Input Slice=inst+desp, alpha=1, type=MAX-10%2026.04 | 12 | — | — | |
| AutoMIATarget Model=LLAMA Adapter, Input Slice=desp, reasoner=DEEPSEEK-V3.2-REASONER2026.04 | 12 | — | — | |
| ModRényiTarget Model=LLaVA, Input Slice=desp, alpha=0.52026.04 | 11.7 | — | — | |
| RényiTarget Model=LLAMA Adapter, Input Slice=inst+desp, alpha=1, type=MAX-10%2026.04 | 11.7 | — | — | |
| AutoMIATarget Model=MiniGPT-4, Input Slice=img, reasoner=DEEPSEEK-V3.2-REASONER2026.04 | 11.7 | — | — | |
| Min-k ProbTarget Model=LLaVA, Input Slice=img, k_threshold=10%2026.04 | 11.3 | — | — | |
| ModRényiTarget Model=LLaVA, Input Slice=desp, alpha=12026.04 | 11.3 | — | — | |
| ModRényiTarget Model=LLaVA, Input Slice=desp, alpha=22026.04 | 11.3 | — | — | |
| ModRényiTarget Model=LLaVA, Input Slice=inst+desp, alpha=22026.04 | 11.3 | — | — | |
| RényiTarget Model=LLaVA, Input Slice=img, alpha=inf, type=MAX-10%2026.04 | 11.3 | — | — | |
| ModRényiTarget Model=LLaVA, Input Slice=inst+desp, alpha=0.52026.04 | 11 | — | — | |
| RényiTarget Model=LLaVA, Input Slice=inst+desp, alpha=inf, type=MAX-10%2026.04 | 11 | — | — | |
| RényiTarget Model=MiniGPT-4, Input Slice=inst, alpha=0.5, type=MAX-0%2026.04 | 10.7 | — | — | |
| RényiTarget Model=LLAMA Adapter, Input Slice=desp, alpha=0.5, type=MAX-0%2026.04 | 10.7 | — | — | |
| ModRényiTarget Model=LLAMA Adapter, Input Slice=desp, alpha=0.52026.04 | 10 | — | — | |
| AutoMIATarget Model=MiniGPT-4, Input Slice=inst, reasoner=DEEPSEEK-V3.2-REASONER2026.04 | 10 | — | — | |
| PerplexityTarget Model=LLAMA Adapter, Input Slice=desp2026.04 | 9.7 | — | — | |
| Min-k ProbTarget Model=LLaVA, Input Slice=img, k_threshold=0%2026.04 | 9.7 | — | — | |
| ModRényiTarget Model=LLAMA Adapter, Input Slice=desp, alpha=22026.04 | 9.7 | — | — | |
| AutoMIATarget Model=MiniGPT-4, Input Slice=desp, reasoner=DEEPSEEK-V3.2-REASONER2026.04 | 9.7 | — | — | |
| Min-k ProbTarget Model=LLaVA, Input Slice=img, k_threshold=20%2026.04 | 9.3 | — | — | |
| RényiTarget Model=LLaVA, Input Slice=inst, alpha=inf, type=MAX-10%2026.04 | 9.3 | — | — | |
| ModRényiTarget Model=LLAMA Adapter, Input Slice=desp, alpha=12026.04 | 9 | — | — | |
| RényiTarget Model=LLaVA, Input Slice=img, alpha=1, type=MAX-10%2026.04 | 9 | — | — | |
| RényiTarget Model=MiniGPT-4, Input Slice=inst, alpha=1, type=MAX-10%2026.04 | 8.7 | — | — | |
| PerplexityTarget Model=LLaVA, Input Slice=inst+desp2026.04 | 8.3 | — | — | |
| Min-k ProbTarget Model=LLaVA, Input Slice=desp, k_threshold=0%2026.04 | 8.3 | — | — | |
| Min-k ProbTarget Model=LLaVA, Input Slice=desp, k_threshold=10%2026.04 | 8.3 | — | — | |
| Min-k ProbTarget Model=LLAMA Adapter, Input Slice=desp, k_threshold=20%2026.04 | 8.3 | — | — | |
| RényiTarget Model=LLaVA, Input Slice=desp, alpha=inf, type=MAX-10%2026.04 | 8.3 | — | — | |
| OpenEvolveLogit Type=Image Logits2026.03 | 8.2 | 72.5 | 1.3 | |
| RényiTarget Model=LLaVA, Input Slice=desp, alpha=0.5, type=MAX-0%2026.04 | 8 | — | — | |
| RényiTarget Model=MiniGPT-4, Input Slice=inst+desp, alpha=0.5, type=MAX-0%2026.04 | 8 | — | — | |
| Max Prob GapTarget Model=LLaVA, Input Slice=inst2026.04 | 7.7 | — | — | |
| ModRényiTarget Model=LLaVA, Input Slice=img, alpha=0.52026.04 | 7.7 | — | — | |
| RényiTarget Model=LLAMA Adapter, Input Slice=desp, alpha=1, type=MAX-10%2026.04 | 7.7 | — | — | |
| ModRényiTarget Model=LLaVA, Input Slice=img, alpha=12026.04 | 7.3 | — | — | |
| ModRényiTarget Model=LLaVA, Input Slice=img, alpha=22026.04 | 7.3 | — | — | |
| AutoMIATarget Model=LLaVA, Input Slice=img, reasoner=DEEPSEEK-V3.2-REASONER2026.04 | 7.3 | — | — | |
| PerplexityTarget Model=LLaVA, Input Slice=img2026.04 | 7 | — | — | |
| Aug-KLTarget Model=LLaVA, Input Slice=inst+desp2026.04 | 6.7 | — | — | |
| Aug-KLTarget Model=LLAMA Adapter, Input Slice=desp2026.04 | 6.7 | — | — | |
| Min-k ProbTarget Model=LLAMA Adapter, Input Slice=desp, k_threshold=0%2026.04 | 6.7 | — | — | |
| RényiTarget Model=MiniGPT-4, Input Slice=img, alpha=0.5, type=MAX-0%2026.04 | 6.7 | — | — | |
| RényiTarget Model=LLaVA, Input Slice=desp, alpha=1, type=MAX-10%2026.04 | 6.7 | — | — | |
| Aug-KLTarget Model=LLAMA Adapter, Input Slice=inst+desp2026.04 | 6.3 | — | — | |
| ModRényiTarget Model=LLaVA, Input Slice=inst+desp, alpha=12026.04 | 6.3 | — | — | |
| RényiTarget Model=MiniGPT-4, Input Slice=img, alpha=1, type=MAX-10%2026.04 | 6.3 | — | — | |
| Max Prob GapTarget Model=LLAMA Adapter, Input Slice=inst2026.04 | 6 | — | — | |
| Min-k ProbTarget Model=LLAMA Adapter, Input Slice=desp, k_threshold=10%2026.04 | 6 | — | — | |
| Max Prob GapTarget Model=LLaVA, Input Slice=img2026.04 | 5.7 | — | — | |
| Aug-KLTarget Model=LLaVA, Input Slice=inst2026.04 | 5.7 | — | — | |
| Aug-KLTarget Model=LLaVA, Input Slice=desp2026.04 | 5.7 | — | — | |
| Min-k ProbTarget Model=MiniGPT-4, Input Slice=inst, k_threshold=0%2026.04 | 5.4 | — | — | |
| Min-k ProbTarget Model=MiniGPT-4, Input Slice=inst, k_threshold=10%2026.04 | 5.4 | — | — | |
| Min-k ProbTarget Model=MiniGPT-4, Input Slice=inst+desp, k_threshold=0%2026.04 | 5.3 | — | — | |
| Max Prob GapTarget Model=MiniGPT-4, Input Slice=inst2026.04 | 5 | — | — | |
| Min-k ProbTarget Model=MiniGPT-4, Input Slice=inst, k_threshold=20%2026.04 | 4.4 | — | — | |
| Aug-KLTarget Model=LLAMA Adapter, Input Slice=inst2026.04 | 4.3 | — | — | |
| RényiTarget Model=LLaVA, Input Slice=img, alpha=0.5, type=MAX-0%2026.04 | 4.3 | — | — | |
| RényiTarget Model=LLAMA Adapter, Input Slice=inst+desp, alpha=0.5, type=MAX-0%2026.04 | 4.3 | — | — | |
| Aug-KLTarget Model=LLaVA, Input Slice=img2026.04 | 4 | — | — | |
| Min-k ProbTarget Model=MiniGPT-4, Input Slice=img, k_threshold=0%2026.04 | 4 | — | — | |
| ModRényiTarget Model=LLAMA Adapter, Input Slice=inst+desp, alpha=22026.04 | 4 | — | — | |
| RényiTarget Model=LLAMA Adapter, Input Slice=inst, alpha=0.5, type=MAX-0%2026.04 | 4 | — | — | |
| Aug-KLTarget Model=MiniGPT-4, Input Slice=img2026.04 | 3.3 | — | — | |
| Min-k ProbTarget Model=MiniGPT-4, Input Slice=img, k_threshold=20%2026.04 | 3.3 | — | — | |
| RényiTarget Model=MiniGPT-4, Input Slice=inst+desp, alpha=1, type=MAX-10%2026.04 | 3.3 | — | — | |
| Max Prob GapTarget Model=MiniGPT-4, Input Slice=img2026.04 | 3 | — | — | |
| Min-k ProbTarget Model=LLAMA Adapter, Input Slice=inst, k_threshold=0%2026.04 | 3 | — | — | |
| Min-k ProbTarget Model=LLAMA Adapter, Input Slice=inst+desp, k_threshold=0%2026.04 | 3 | — | — | |
| ModRényiTarget Model=MiniGPT-4, Input Slice=img, alpha=22026.04 | 3 | — | — | |
| Max Prob GapTarget Model=MiniGPT-4, Input Slice=desp2026.04 | 2.7 | — | — | |
| Aug-KLTarget Model=MiniGPT-4, Input Slice=inst2026.04 | 2.7 | — | — | |
| Min-k ProbTarget Model=MiniGPT-4, Input Slice=desp, k_threshold=0%2026.04 | 2.7 | — | — | |
| Min-k ProbTarget Model=MiniGPT-4, Input Slice=img, k_threshold=10%2026.04 | 2.7 | — | — | |
| ModRényiTarget Model=MiniGPT-4, Input Slice=img, alpha=0.52026.04 | 2.7 | — | — | |
| ModRényiTarget Model=MiniGPT-4, Input Slice=inst, alpha=0.52026.04 | 2.7 | — | — | |
| ModRényiTarget Model=LLAMA Adapter, Input Slice=inst+desp, alpha=0.52026.04 | 2.7 | — | — |