Membership Inference Attack on DALL·E (test)
29.4TPR @ 5% FPRAutoMIA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AutoMIAAgent=DEEPSEEK-V3.2-REASONER, Target Model=LLaVA, Input Type=img2026.04 | 29.4 | — | |
| Renyialpha=1, Max=10%, Target Model=LLaVA, Input Type=img2026.04 | 22.3 | — | |
| AutoMIAAgent=DEEPSEEK-V3.2-REASONER, Target Model=LLaVA, Input Type=inst2026.04 | 17.6 | — | |
| AutoMIAAgent=DEEPSEEK-V3.2-REASONER, Target Model=LLaMA Adapter, Input Type=inst2026.04 | 15.9 | — | |
| Min-k Probk=10%, Target Model=LLaVA, Input Type=img2026.04 | 13.5 | — | |
| Min-k Probk=0%, Target Model=LLaVA, Input Type=img2026.04 | 13.2 | — | |
| PerplexityTarget Model=MiniGPT-4, Input Type=img2026.04 | 12.8 | — | |
| Renyialpha=inf, Max=100%, Target Model=MiniGPT-4, Input Type=img2026.04 | 12.8 | — | |
| Min-k Probk=20%, Target Model=LLaVA, Input Type=img2026.04 | 12.5 | — | |
| AutoMIAAgent=DEEPSEEK-V3.2-REASONER, Target Model=LLaVA, Input Type=inst+desp2026.04 | 12.5 | — | |
| Max Prob GapTarget Model=MiniGPT-4, Input Type=desp2026.04 | 11.5 | — | |
| AutoMIAAgent=DEEPSEEK-V3.2-REASONER, Target Model=LLaVA, Input Type=desp2026.04 | 11.5 | — | |
| Max Prob GapTarget Model=MiniGPT-4, Input Type=inst+desp2026.04 | 10.8 | — | |
| Max Prob GapTarget Model=LLaMA Adapter, Input Type=inst2026.04 | 10.8 | — | |
| Aug-KLTarget Model=LLaMA Adapter, Input Type=inst2026.04 | 10.8 | — | |
| Aug-KLTarget Model=MiniGPT-4, Input Type=inst2026.04 | 10.5 | — | |
| Aug-KLTarget Model=LLaMA Adapter, Input Type=desp2026.04 | 9.8 | — | |
| Aug-KLTarget Model=LLaMA Adapter, Input Type=inst+desp2026.04 | 9.8 | — | |
| Renyialpha=0.5, Max=0%, Target Model=LLaVA, Input Type=img2026.04 | 9.8 | — | |
| Renyialpha=2, Max=100%, Target Model=LLaVA, Input Type=inst+desp2026.04 | 9.5 | — | |
| AutoMIAAgent=DEEPSEEK-V3.2-REASONER, Target Model=LLaMA Adapter, Input Type=inst+desp2026.04 | 9.1 | — | |
| Max Prob GapTarget Model=MiniGPT-4, Input Type=img2026.04 | 8.8 | — | |
| Max Prob GapTarget Model=LLaVA, Input Type=inst2026.04 | 8.5 | — | |
| Aug-KLTarget Model=LLaVA, Input Type=desp2026.04 | 8.5 | — | |
| Max Prob GapTarget Model=LLaVA, Input Type=desp2026.04 | 8.1 | — | |
| Aug-KLTarget Model=LLaVA, Input Type=inst+desp2026.04 | 8.1 | — | |
| Aug-KLTarget Model=MiniGPT-4, Input Type=inst+desp2026.04 | 8.1 | — | |
| AutoMIAAgent=DEEPSEEK-V3.2-REASONER, Target Model=MiniGPT-4, Input Type=inst+desp2026.04 | 8.1 | — | |
| PerplexityTarget Model=LLaVA, Input Type=desp2026.04 | 7.8 | — | |
| AutoMIAAgent=DEEPSEEK-V3.2-REASONER, Target Model=LLaMA Adapter, Input Type=desp2026.04 | 7.8 | — | |
| AutoMIAAgent=DEEPSEEK-V3.2-REASONER, Target Model=MiniGPT-4, Input Type=desp2026.04 | 7.4 | — | |
| AutoMIAAgent=DEEPSEEK-V3.2-REASONER, Target Model=MiniGPT-4, Input Type=img2026.04 | 7.1 | — | |
| AutoMIAAgent=DEEPSEEK-V3.2-REASONER, Target Model=MiniGPT-4, Input Type=inst2026.04 | 6.8 | — | |
| Max Prob GapTarget Model=LLaVA, Input Type=inst+desp2026.04 | 6.1 | — | |
| PerplexityTarget Model=MiniGPT-4, Input Type=desp2026.04 | 5.7 | — | |
| PerplexityTarget Model=LLaVA, Input Type=inst+desp2026.04 | 5.4 | — | |
| Aug-KLTarget Model=LLaVA, Input Type=inst2026.04 | 5.4 | — | |
| PerplexityTarget Model=MiniGPT-4, Input Type=inst+desp2026.04 | 5.1 | — | |
| PerplexityTarget Model=LLaMA Adapter, Input Type=inst2026.04 | 5.1 | — | |
| PerplexityTarget Model=LLaMA Adapter, Input Type=desp2026.04 | 5.1 | — | |
| Aug-KLTarget Model=MiniGPT-4, Input Type=img2026.04 | 4.7 | — | |
| PerplexityTarget Model=MiniGPT-4, Input Type=inst2026.04 | 4.4 | — | |
| PerplexityTarget Model=LLaMA Adapter, Input Type=inst+desp2026.04 | 4.4 | — | |
| Max Prob GapTarget Model=LLaMA Adapter, Input Type=inst+desp2026.04 | 4.1 | — | |
| Aug-KLTarget Model=MiniGPT-4, Input Type=desp2026.04 | 4.1 | — | |
| Max Prob GapTarget Model=LLaVA, Input Type=img2026.04 | 3.7 | — | |
| Max Prob GapTarget Model=LLaMA Adapter, Input Type=desp2026.04 | 3.4 | — | |
| PerplexityTarget Model=LLaVA, Input Type=inst2026.04 | 2.7 | — | |
| Max Prob GapTarget Model=MiniGPT-4, Input Type=inst2026.04 | 2.7 | — | |
| Aug-KLTarget Model=LLaVA, Input Type=img2026.04 | 2.7 | — | |
| ModRenyialpha=1, Target Model=LLaVA, Input Type=img2026.04 | 2.7 | — | |
| PerplexityTarget Model=LLaVA, Input Type=img2026.04 | 2 | — | |
| ModRenyialpha=0.5, Target Model=LLaVA, Input Type=img2026.04 | 1.7 | — | |
| ModRenyialpha=2, Target Model=LLaVA, Input Type=img2026.04 | 1.4 | — | |
| Aug-KLVictim Model=LLaVA, Input Slice=IMG2026.04 | — | 0.432 | |
| Aug-KLVictim Model=LLaVA, Input Slice=INST2026.04 | — | 0.462 | |
| Aug-KLVictim Model=LLaVA, Input Slice=DESP2026.04 | — | 0.523 | |
| Aug-KLVictim Model=LLaVA, Input Slice=INST+DESP2026.04 | — | 0.504 | |
| AutoMIAVictim Model=LLaVA, Agent Backbone=DeepSeek-V3.2-Reasoner, Input Slice=IMG2026.04 | — | 0.787 | |
| AutoMIAVictim Model=LLaVA, Agent Backbone=DeepSeek-V3.2-Reasoner, Input Slice=INST2026.04 | — | 0.663 | |
| AutoMIAVictim Model=LLaVA, Agent Backbone=DeepSeek-V3.2-Reasoner, Input Slice=DESP2026.04 | — | 0.598 | |
| AutoMIAVictim Model=LLaVA, Agent Backbone=DeepSeek-V3.2-Reasoner, Input Slice=INST+DESP2026.04 | — | 0.653 | |
| Max Prob GapVictim Model=LLaVA, Input Slice=IMG2026.04 | — | 0.529 | |
| Max Prob GapVictim Model=LLaVA, Input Slice=INST2026.04 | — | 0.578 | |
| Max Prob GapVictim Model=LLaVA, Input Slice=DESP2026.04 | — | 0.598 | |
| Max Prob GapVictim Model=LLaVA, Input Slice=INST+DESP2026.04 | — | 0.603 | |
| Min-k ProbVictim Model=LLaVA, Pooling Strategy=Min-0%, Input Slice=INST2026.04 | — | 0.481 | |
| Min-k ProbVictim Model=LLaVA, Pooling Strategy=Min-0%, Input Slice=DESP2026.04 | — | 0.556 | |
| Min-k ProbVictim Model=LLaVA, Pooling Strategy=Min-0%, Input Slice=INST+DESP2026.04 | — | 0.481 | |
| Min-k ProbVictim Model=LLaVA, Pooling Strategy=Min-10%, Input Slice=INST2026.04 | — | 0.481 | |
| Min-k ProbVictim Model=LLaVA, Pooling Strategy=Min-10%, Input Slice=DESP2026.04 | — | 0.561 | |
| Min-k ProbVictim Model=LLaVA, Pooling Strategy=Min-10%, Input Slice=INST+DESP2026.04 | — | 0.424 | |
| Min-k ProbVictim Model=LLaVA, Pooling Strategy=Min-20%, Input Slice=INST2026.04 | — | 0.434 | |
| Min-k ProbVictim Model=LLaVA, Pooling Strategy=Min-20%, Input Slice=DESP2026.04 | — | 0.56 | |
| Min-k ProbVictim Model=LLaVA, Pooling Strategy=Min-20%, Input Slice=INST+DESP2026.04 | — | 0.352 | |
| ModRényi GapVictim Model=LLaVA, Alpha=0.5, Input Slice=INST2026.04 | — | 0.359 | |
| ModRényi GapVictim Model=LLaVA, Alpha=0.5, Input Slice=DESP2026.04 | — | 0.563 | |
| ModRényi GapVictim Model=LLaVA, Alpha=0.5, Input Slice=INST+DESP2026.04 | — | 0.525 | |
| ModRényi GapVictim Model=LLaVA, Alpha=1, Input Slice=INST2026.04 | — | 0.341 | |
| ModRényi GapVictim Model=LLaVA, Alpha=1, Input Slice=DESP2026.04 | — | 0.563 | |
| ModRényi GapVictim Model=LLaVA, Alpha=1, Input Slice=INST+DESP2026.04 | — | 0.425 | |
| ModRényi GapVictim Model=LLaVA, Alpha=2, Input Slice=INST2026.04 | — | 0.383 | |
| ModRényi GapVictim Model=LLaVA, Alpha=2, Input Slice=DESP2026.04 | — | 0.564 | |
| ModRényi GapVictim Model=LLaVA, Alpha=2, Input Slice=INST+DESP2026.04 | — | 0.539 | |
| PerplexityVictim Model=LLaVA, Input Slice=INST2026.04 | — | 0.337 | |
| PerplexityVictim Model=LLaVA, Input Slice=DESP2026.04 | — | 0.567 | |
| PerplexityVictim Model=LLaVA, Input Slice=INST+DESP2026.04 | — | 0.448 | |
| RényiVictim Model=LLaVA, Alpha=0.5, Pooling Strategy=Max-0%, Input Slice=IMG2026.04 | — | 0.553 | |
| RényiVictim Model=LLaVA, Alpha=0.5, Pooling Strategy=Max-0%, Input Slice=INST2026.04 | — | 0.598 | |
| RényiVictim Model=LLaVA, Alpha=0.5, Pooling Strategy=Max-0%, Input Slice=DESP2026.04 | — | 0.557 | |
| RényiVictim Model=LLaVA, Alpha=0.5, Pooling Strategy=Max-0%, Input Slice=INST+DESP2026.04 | — | 0.598 | |
| RényiVictim Model=LLaVA, Alpha=1, Pooling Strategy=Max-10%, Input Slice=IMG2026.04 | — | 0.666 | |
| RényiVictim Model=LLaVA, Alpha=2, Pooling Strategy=Max-10%, Input Slice=IMG2026.04 | — | 0.705 | |
| RényiVictim Model=LLaVA, Alpha=inf, Pooling Strategy=Max-10%, Input Slice=IMG2026.04 | — | 0.698 |