Membership Inference Attack on DALL-E
72.3AccuracyAutoMIA (Agent)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| AutoMIA (Agent)Backbone=LLaVA, Input Slice=img, Reasoner=DeepSeek-V3.2-Reasoner2026.04 | 72.3 | — | — | — | |
| Renyi (alpha=2)Backbone=LLaVA, Input Slice=img, Max-k=10%2026.04 | 66.7 | — | — | — | |
| Min-k ProbBackbone=LLaVA, Input Slice=img, k=10%2026.04 | 65.9 | — | — | — | |
| Renyi (alpha=inf)Backbone=LLaVA, Input Slice=img, Max-k=10%2026.04 | 65.9 | — | — | — | |
| Min-k ProbBackbone=LLaVA, Input Slice=img, k=20%2026.04 | 63.7 | — | — | — | |
| AutoMIA (Agent)Backbone=LLaVA, Input Slice=inst, Reasoner=DeepSeek-V3.2-Reasoner2026.04 | 63.3 | — | — | — | |
| Renyi (alpha=0.5)Backbone=LLaVA, Input Slice=inst+desp, Max-k=10%2026.04 | 61.5 | — | — | — | |
| Min-k ProbBackbone=LLaVA, Input Slice=img, k=0%2026.04 | 61.3 | — | — | — | |
| Renyi (alpha=1)Backbone=LLaVA, Input Slice=inst, Max-k=100%2026.04 | 60.6 | — | — | — | |
| AutoMIA (Agent)Backbone=LLaMA Adapter, Input Slice=inst, Reasoner=DeepSeek-V3.2-Reasoner2026.04 | 60 | — | — | — | |
| Max Prob GapBackbone=LLaVA, Input Slice=inst+desp2026.04 | 59.3 | — | — | — | |
| Max Prob GapBackbone=LLaVA, Input Slice=desp2026.04 | 59.1 | — | — | — | |
| AutoMIA (Agent)Backbone=MiniGPT-4, Input Slice=img, Reasoner=DeepSeek-V3.2-Reasoner2026.04 | 58.1 | — | — | — | |
| AutoMIA (Agent)Backbone=MiniGPT-4, Input Slice=inst+desp, Reasoner=DeepSeek-V3.2-Reasoner2026.04 | 57.8 | — | — | — | |
| Max Prob GapBackbone=LLaVA, Input Slice=inst2026.04 | 57.1 | — | — | — | |
| PerplexityBackbone=LLaVA, Input Slice=desp2026.04 | 56.9 | — | — | — | |
| PerplexityBackbone=MiniGPT-4, Input Slice=img2026.04 | 56.6 | — | — | — | |
| AutoMIA (Agent)Backbone=LLaMA Adapter, Input Slice=inst+desp, Reasoner=DeepSeek-V3.2-Reasoner2026.04 | 56.6 | — | — | — | |
| AutoMIA (Agent)Backbone=LLaMA Adapter, Input Slice=desp, Reasoner=DeepSeek-V3.2-Reasoner2026.04 | 56.1 | — | — | — | |
| PerplexityBackbone=LLaVA, Input Slice=img2026.04 | 54.9 | — | — | — | |
| ModRenyiBackbone=LLaVA, Input Slice=img, alpha=0.52026.04 | 54.4 | — | — | — | |
| Max Prob GapBackbone=LLaVA, Input Slice=img2026.04 | 53.7 | — | — | — | |
| PerplexityBackbone=LLaMA Adapter, Input Slice=inst2026.04 | 51.1 | — | — | — | |
| PerplexityBackbone=LLaVA, Input Slice=inst+desp2026.04 | 50.7 | — | — | — | |
| PerplexityBackbone=LLaVA, Input Slice=inst2026.04 | 50.5 | — | — | — | |
| Aug-KLBackbone=LLaVA, Input Slice=img2026.04 | 50 | — | — | — | |
| AutoMIALogit Type=Image Logits2026.03 | — | 0.752 | 5.4 | 30.4 | |
| AutoMIALogit Type=Text Logits2026.03 | — | 0.705 | 0.7 | 20.9 | |
| Li et al., 2024Logit Type=Image Logits2026.03 | — | 0.594 | 5.4 | 13.5 | |
| Li et al., 2024Logit Type=Text Logits2026.03 | — | 0.605 | 0.7 | 14.2 | |
| OpenEvolveLogit Type=Image Logits2026.03 | — | 0.612 | 2.7 | 13.5 | |
| OpenEvolveLogit Type=Text Logits2026.03 | — | 0.575 | 0.7 | 9.7 |