List your company
ResearchFormerly WizWand

Membership Inference on VL-MIA DALL·E

89.1AUROCDistractMIA
49.99660.14870.380.452May 12, 2026
Updated 5mo ago

Evaluation results

MethodLinks
DistractMIATarget Model=LLaVA, K (Set Size)=302026.0589.1
PerplexityTarget Model=LLaVA, K (Set Size)=302026.0587.6
Max-Prob-GapTarget Model=LLaVA, K (Set Size)=302026.0586.5
ModRényiTarget Model=LLaVA, K (Set Size)=302026.0578.9
Min-K%Target Model=LLaVA, K (Set Size)=302026.0577.7
MaxRényi-K%Target Model=LLaVA, K (Set Size)=302026.0576.4
DistractMIATarget Model=LLaVA, K (Set Size)=102026.0575.3
DistractMIATarget Model=Average, K (Set Size)=302026.0575.3
Max-Prob-GapTarget Model=LLaVA, K (Set Size)=102026.0574.5
PerplexityTarget Model=LLaVA, K (Set Size)=102026.0574.2
Max-Prob-GapTarget Model=Average, K (Set Size)=302026.0572.2
PerplexityTarget Model=Average, K (Set Size)=302026.0570.8
MaxRényi-K%Target Model=Average, K (Set Size)=302026.0569.9
DistractMIATarget Model=MiniGPT-4, K (Set Size)=302026.0568.4
DistractMIATarget Model=LLaMA2-A, K (Set Size)=302026.0568.3
PerplexityTarget Model=MiniGPT-4, K (Set Size)=302026.0568.1
Aug-KLTarget Model=LLaMA2-A, K (Set Size)=302026.0567.9
ModRényiTarget Model=Average, K (Set Size)=302026.0567.9
MaxRényi-K%Target Model=LLaMA2-A, K (Set Size)=302026.0567.5
Max-Prob-GapTarget Model=LLaMA2-A, K (Set Size)=302026.0567.4
Min-K%Target Model=LLaVA, K (Set Size)=102026.0567.3
DistractMIATarget Model=Average, K (Set Size)=102026.0566.8
MaxRényi-K%Target Model=LLaVA, K (Set Size)=102026.0566
MaxRényi-K%Target Model=MiniGPT-4, K (Set Size)=302026.0565.7
ModRényiTarget Model=LLaVA, K (Set Size)=102026.0564.9
DistractMIATarget Model=MiniGPT-4, K (Set Size)=102026.0564.6
Min-K%Target Model=Average, K (Set Size)=302026.0564.5
Aug-KLTarget Model=MiniGPT-4, K (Set Size)=102026.0564
ModRényiTarget Model=MiniGPT-4, K (Set Size)=302026.0564
MaxRényi-K%Target Model=MiniGPT-4, K (Set Size)=102026.0564
Max-Prob-GapTarget Model=Average, K (Set Size)=102026.0563.9
Aug-KLTarget Model=Average, K (Set Size)=302026.0563.7
PerplexityTarget Model=Average, K (Set Size)=102026.0563.5
PerplexityTarget Model=MiniGPT-4, K (Set Size)=102026.0563.3
Aug-KLTarget Model=MiniGPT-4, K (Set Size)=302026.0562.8
Max-Prob-GapTarget Model=MiniGPT-4, K (Set Size)=302026.0562.8
MaxRényi-K%Target Model=Average, K (Set Size)=102026.0562.5
ModRényiTarget Model=MiniGPT-4, K (Set Size)=102026.0561.2
ModRényiTarget Model=LLaMA2-A, K (Set Size)=302026.0560.7
DistractMIATarget Model=LLaMA2-A, K (Set Size)=102026.0560.6
Aug-KLTarget Model=LLaVA, K (Set Size)=302026.0560.4
Max-Prob-GapTarget Model=LLaMA2-A, K (Set Size)=102026.0560.3
ModRényiTarget Model=Average, K (Set Size)=102026.0559.9
Min-K%Target Model=MiniGPT-4, K (Set Size)=302026.0558.9
Min-K%Target Model=Average, K (Set Size)=102026.0558.2
DistractMIATarget Model=LLaVA, K (Set Size)=12026.0558.2
Aug-KLTarget Model=Average, K (Set Size)=102026.0558
Aug-KLTarget Model=LLaMA2-A, K (Set Size)=102026.0557.4
MaxRényi-K%Target Model=LLaMA2-A, K (Set Size)=102026.0557.4
Max-Prob-GapTarget Model=LLaVA, K (Set Size)=12026.0556.9
Max-Prob-GapTarget Model=MiniGPT-4, K (Set Size)=102026.0556.9
Min-K%Target Model=LLaMA2-A, K (Set Size)=302026.0556.9
PerplexityTarget Model=LLaMA2-A, K (Set Size)=302026.0556.7
KCMPTarget Model=MiniGPT-4, K (Set Size)=302026.0556.3
PerplexityTarget Model=LLaVA, K (Set Size)=12026.0555.8
Min-K%Target Model=LLaVA, K (Set Size)=12026.0555.8
KCMPTarget Model=MiniGPT-4, K (Set Size)=102026.0555.4
DistractMIATarget Model=Average, K (Set Size)=12026.0555.1
MaxRényi-K%Target Model=LLaVA, K (Set Size)=12026.0554.8
Min-K%Target Model=MiniGPT-4, K (Set Size)=102026.0554.5
Max-Prob-GapTarget Model=Average, K (Set Size)=12026.0554.4
Image Infer (Rouge)Target Model=MiniGPT-4, K (Set Size)=302026.0554.3
KCMPTarget Model=Average, K (Set Size)=302026.0554.2
DistractMIATarget Model=MiniGPT-4, K (Set Size)=12026.0554.2
KCMPTarget Model=LLaMA2-A, K (Set Size)=302026.0554.1
ModRényiTarget Model=LLaVA, K (Set Size)=12026.0554
ModRényiTarget Model=MiniGPT-4, K (Set Size)=12026.0553.9
Image Infer (Rouge)Target Model=LLaMA2-A, K (Set Size)=302026.0553.8
Max-Prob-GapTarget Model=MiniGPT-4, K (Set Size)=12026.0553.7
ModRényiTarget Model=LLaMA2-A, K (Set Size)=102026.0553.6
Image Infer (Rouge)Target Model=LLaMA2-A, K (Set Size)=102026.0553.6
KCMPTarget Model=Average, K (Set Size)=102026.0553.5
Aug-KLTarget Model=MiniGPT-4, K (Set Size)=12026.0553.4
MaxRényi-K%Target Model=Average, K (Set Size)=12026.0553.4
Image Infer (Rouge)Target Model=Average, K (Set Size)=302026.0553.4
Min-K%Target Model=Average, K (Set Size)=12026.0553.3
PerplexityTarget Model=LLaMA2-A, K (Set Size)=102026.0553.2
PerplexityTarget Model=Average, K (Set Size)=12026.0553
ModRényiTarget Model=Average, K (Set Size)=12026.0553
DistractMIATarget Model=LLaMA2-A, K (Set Size)=12026.0553
Min-K%Target Model=LLaMA2-A, K (Set Size)=102026.0552.8
MaxRényi-K%Target Model=LLaMA2-A, K (Set Size)=12026.0552.8
Max-Prob-GapTarget Model=LLaMA2-A, K (Set Size)=12026.0552.7
Min-K%Target Model=MiniGPT-4, K (Set Size)=12026.0552.7
KCMPTarget Model=LLaMA2-A, K (Set Size)=102026.0552.7
Image Infer (MPNet)Target Model=MiniGPT-4, K (Set Size)=302026.0552.6
PerplexityTarget Model=MiniGPT-4, K (Set Size)=12026.0552.5
Aug-KLTarget Model=LLaVA, K (Set Size)=102026.0552.5
MaxRényi-K%Target Model=MiniGPT-4, K (Set Size)=12026.0552.5
KCMPTarget Model=LLaVA, K (Set Size)=102026.0552.4
Aug-KLTarget Model=Average, K (Set Size)=12026.0552.3
Aug-KLTarget Model=LLaVA, K (Set Size)=12026.0552.2
Image Infer (MPNet)Target Model=LLaMA2-A, K (Set Size)=102026.0552.2
KCMPTarget Model=LLaVA, K (Set Size)=302026.0552.2
Image Infer (Rouge)Target Model=LLaVA, K (Set Size)=302026.0552.1
Image Infer (Rouge)Target Model=MiniGPT-4, K (Set Size)=12026.0552.1
Image Infer (MPNet)Target Model=LLaMA2-A, K (Set Size)=12026.0551.9
KCMPTarget Model=LLaMA2-A, K (Set Size)=12026.0551.8
Image Infer (Rouge)Target Model=Average, K (Set Size)=102026.0551.7
Image Infer (MPNet)Target Model=LLaVA, K (Set Size)=302026.0551.5