Membership Inference on VL-MIA DALL·E
89.1AUROCDistractMIA
Evaluation Results
| Method | Links | |
|---|---|---|
| DistractMIATarget Model=LLaVA, K (Set Size)=302026.05 | 89.1 | |
| PerplexityTarget Model=LLaVA, K (Set Size)=302026.05 | 87.6 | |
| Max-Prob-GapTarget Model=LLaVA, K (Set Size)=302026.05 | 86.5 | |
| ModRényiTarget Model=LLaVA, K (Set Size)=302026.05 | 78.9 | |
| Min-K%Target Model=LLaVA, K (Set Size)=302026.05 | 77.7 | |
| MaxRényi-K%Target Model=LLaVA, K (Set Size)=302026.05 | 76.4 | |
| DistractMIATarget Model=LLaVA, K (Set Size)=102026.05 | 75.3 | |
| DistractMIATarget Model=Average, K (Set Size)=302026.05 | 75.3 | |
| Max-Prob-GapTarget Model=LLaVA, K (Set Size)=102026.05 | 74.5 | |
| PerplexityTarget Model=LLaVA, K (Set Size)=102026.05 | 74.2 | |
| Max-Prob-GapTarget Model=Average, K (Set Size)=302026.05 | 72.2 | |
| PerplexityTarget Model=Average, K (Set Size)=302026.05 | 70.8 | |
| MaxRényi-K%Target Model=Average, K (Set Size)=302026.05 | 69.9 | |
| DistractMIATarget Model=MiniGPT-4, K (Set Size)=302026.05 | 68.4 | |
| DistractMIATarget Model=LLaMA2-A, K (Set Size)=302026.05 | 68.3 | |
| PerplexityTarget Model=MiniGPT-4, K (Set Size)=302026.05 | 68.1 | |
| Aug-KLTarget Model=LLaMA2-A, K (Set Size)=302026.05 | 67.9 | |
| ModRényiTarget Model=Average, K (Set Size)=302026.05 | 67.9 | |
| MaxRényi-K%Target Model=LLaMA2-A, K (Set Size)=302026.05 | 67.5 | |
| Max-Prob-GapTarget Model=LLaMA2-A, K (Set Size)=302026.05 | 67.4 | |
| Min-K%Target Model=LLaVA, K (Set Size)=102026.05 | 67.3 | |
| DistractMIATarget Model=Average, K (Set Size)=102026.05 | 66.8 | |
| MaxRényi-K%Target Model=LLaVA, K (Set Size)=102026.05 | 66 | |
| MaxRényi-K%Target Model=MiniGPT-4, K (Set Size)=302026.05 | 65.7 | |
| ModRényiTarget Model=LLaVA, K (Set Size)=102026.05 | 64.9 | |
| DistractMIATarget Model=MiniGPT-4, K (Set Size)=102026.05 | 64.6 | |
| Min-K%Target Model=Average, K (Set Size)=302026.05 | 64.5 | |
| Aug-KLTarget Model=MiniGPT-4, K (Set Size)=102026.05 | 64 | |
| ModRényiTarget Model=MiniGPT-4, K (Set Size)=302026.05 | 64 | |
| MaxRényi-K%Target Model=MiniGPT-4, K (Set Size)=102026.05 | 64 | |
| Max-Prob-GapTarget Model=Average, K (Set Size)=102026.05 | 63.9 | |
| Aug-KLTarget Model=Average, K (Set Size)=302026.05 | 63.7 | |
| PerplexityTarget Model=Average, K (Set Size)=102026.05 | 63.5 | |
| PerplexityTarget Model=MiniGPT-4, K (Set Size)=102026.05 | 63.3 | |
| Aug-KLTarget Model=MiniGPT-4, K (Set Size)=302026.05 | 62.8 | |
| Max-Prob-GapTarget Model=MiniGPT-4, K (Set Size)=302026.05 | 62.8 | |
| MaxRényi-K%Target Model=Average, K (Set Size)=102026.05 | 62.5 | |
| ModRényiTarget Model=MiniGPT-4, K (Set Size)=102026.05 | 61.2 | |
| ModRényiTarget Model=LLaMA2-A, K (Set Size)=302026.05 | 60.7 | |
| DistractMIATarget Model=LLaMA2-A, K (Set Size)=102026.05 | 60.6 | |
| Aug-KLTarget Model=LLaVA, K (Set Size)=302026.05 | 60.4 | |
| Max-Prob-GapTarget Model=LLaMA2-A, K (Set Size)=102026.05 | 60.3 | |
| ModRényiTarget Model=Average, K (Set Size)=102026.05 | 59.9 | |
| Min-K%Target Model=MiniGPT-4, K (Set Size)=302026.05 | 58.9 | |
| Min-K%Target Model=Average, K (Set Size)=102026.05 | 58.2 | |
| DistractMIATarget Model=LLaVA, K (Set Size)=12026.05 | 58.2 | |
| Aug-KLTarget Model=Average, K (Set Size)=102026.05 | 58 | |
| Aug-KLTarget Model=LLaMA2-A, K (Set Size)=102026.05 | 57.4 | |
| MaxRényi-K%Target Model=LLaMA2-A, K (Set Size)=102026.05 | 57.4 | |
| Max-Prob-GapTarget Model=LLaVA, K (Set Size)=12026.05 | 56.9 | |
| Max-Prob-GapTarget Model=MiniGPT-4, K (Set Size)=102026.05 | 56.9 | |
| Min-K%Target Model=LLaMA2-A, K (Set Size)=302026.05 | 56.9 | |
| PerplexityTarget Model=LLaMA2-A, K (Set Size)=302026.05 | 56.7 | |
| KCMPTarget Model=MiniGPT-4, K (Set Size)=302026.05 | 56.3 | |
| PerplexityTarget Model=LLaVA, K (Set Size)=12026.05 | 55.8 | |
| Min-K%Target Model=LLaVA, K (Set Size)=12026.05 | 55.8 | |
| KCMPTarget Model=MiniGPT-4, K (Set Size)=102026.05 | 55.4 | |
| DistractMIATarget Model=Average, K (Set Size)=12026.05 | 55.1 | |
| MaxRényi-K%Target Model=LLaVA, K (Set Size)=12026.05 | 54.8 | |
| Min-K%Target Model=MiniGPT-4, K (Set Size)=102026.05 | 54.5 | |
| Max-Prob-GapTarget Model=Average, K (Set Size)=12026.05 | 54.4 | |
| Image Infer (Rouge)Target Model=MiniGPT-4, K (Set Size)=302026.05 | 54.3 | |
| KCMPTarget Model=Average, K (Set Size)=302026.05 | 54.2 | |
| DistractMIATarget Model=MiniGPT-4, K (Set Size)=12026.05 | 54.2 | |
| KCMPTarget Model=LLaMA2-A, K (Set Size)=302026.05 | 54.1 | |
| ModRényiTarget Model=LLaVA, K (Set Size)=12026.05 | 54 | |
| ModRényiTarget Model=MiniGPT-4, K (Set Size)=12026.05 | 53.9 | |
| Image Infer (Rouge)Target Model=LLaMA2-A, K (Set Size)=302026.05 | 53.8 | |
| Max-Prob-GapTarget Model=MiniGPT-4, K (Set Size)=12026.05 | 53.7 | |
| ModRényiTarget Model=LLaMA2-A, K (Set Size)=102026.05 | 53.6 | |
| Image Infer (Rouge)Target Model=LLaMA2-A, K (Set Size)=102026.05 | 53.6 | |
| KCMPTarget Model=Average, K (Set Size)=102026.05 | 53.5 | |
| Aug-KLTarget Model=MiniGPT-4, K (Set Size)=12026.05 | 53.4 | |
| MaxRényi-K%Target Model=Average, K (Set Size)=12026.05 | 53.4 | |
| Image Infer (Rouge)Target Model=Average, K (Set Size)=302026.05 | 53.4 | |
| Min-K%Target Model=Average, K (Set Size)=12026.05 | 53.3 | |
| PerplexityTarget Model=LLaMA2-A, K (Set Size)=102026.05 | 53.2 | |
| PerplexityTarget Model=Average, K (Set Size)=12026.05 | 53 | |
| ModRényiTarget Model=Average, K (Set Size)=12026.05 | 53 | |
| DistractMIATarget Model=LLaMA2-A, K (Set Size)=12026.05 | 53 | |
| Min-K%Target Model=LLaMA2-A, K (Set Size)=102026.05 | 52.8 | |
| MaxRényi-K%Target Model=LLaMA2-A, K (Set Size)=12026.05 | 52.8 | |
| Max-Prob-GapTarget Model=LLaMA2-A, K (Set Size)=12026.05 | 52.7 | |
| Min-K%Target Model=MiniGPT-4, K (Set Size)=12026.05 | 52.7 | |
| KCMPTarget Model=LLaMA2-A, K (Set Size)=102026.05 | 52.7 | |
| Image Infer (MPNet)Target Model=MiniGPT-4, K (Set Size)=302026.05 | 52.6 | |
| PerplexityTarget Model=MiniGPT-4, K (Set Size)=12026.05 | 52.5 | |
| Aug-KLTarget Model=LLaVA, K (Set Size)=102026.05 | 52.5 | |
| MaxRényi-K%Target Model=MiniGPT-4, K (Set Size)=12026.05 | 52.5 | |
| KCMPTarget Model=LLaVA, K (Set Size)=102026.05 | 52.4 | |
| Aug-KLTarget Model=Average, K (Set Size)=12026.05 | 52.3 | |
| Aug-KLTarget Model=LLaVA, K (Set Size)=12026.05 | 52.2 | |
| Image Infer (MPNet)Target Model=LLaMA2-A, K (Set Size)=102026.05 | 52.2 | |
| KCMPTarget Model=LLaVA, K (Set Size)=302026.05 | 52.2 | |
| Image Infer (Rouge)Target Model=LLaVA, K (Set Size)=302026.05 | 52.1 | |
| Image Infer (Rouge)Target Model=MiniGPT-4, K (Set Size)=12026.05 | 52.1 | |
| Image Infer (MPNet)Target Model=LLaMA2-A, K (Set Size)=12026.05 | 51.9 | |
| KCMPTarget Model=LLaMA2-A, K (Set Size)=12026.05 | 51.8 | |
| Image Infer (Rouge)Target Model=Average, K (Set Size)=102026.05 | 51.7 | |
| Image Infer (MPNet)Target Model=LLaVA, K (Set Size)=302026.05 | 51.5 |