Membership Inference Attack on VLLM instruction-tuning texts length 64 (test)
0.514Rényi (α=0.5) Max Prob (0%)MaxRényi-K%
Evaluation results
| Method | Links | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MaxRényi-K%Target Model=LLaVA2024.11 | 0.514 | — | — | — | — | — | — | — | — | — | — | 0.776 | 0.835 | 0.579 | 0.809 | 0.75 | 0.625 | 0.787 | 0.709 | 0.638 | 0.763 | 0.694 | |
| MaxRényi-K%Target Model=MiniGPT-42024.11 | 0.511 | — | — | — | — | — | — | — | — | — | — | 0.512 | 0.945 | 0.508 | 0.781 | 0.911 | 0.69 | 0.843 | 0.902 | 0.765 | 0.843 | 0.896 | |
| MaxRényi-K%Target Model=LLaMA Adapter2024.11 | 0.51 | — | — | — | — | — | — | — | — | — | — | 0.836 | 0.835 | 0.603 | 0.828 | 0.762 | 0.635 | 0.795 | 0.716 | 0.616 | 0.755 | 0.695 | |
| MaxRényi-K%Target Model=LLaVA, α=0.5, K=0%2024.11 | 0.51 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Max_Prob_GapTarget Model=LLaVA2024.11 | — | — | — | — | — | — | — | 0.562 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MaxRényi-K%Target Model=LLaVA, α=0.5, K=10%2024.11 | — | — | — | — | — | — | — | — | — | — | — | 0.79 | — | — | — | — | — | — | — | — | — | — | |
| MaxRényi-K%Target Model=LLaVA, α=0.5, K=100%2024.11 | — | — | — | — | — | — | — | — | — | — | — | — | 0.847 | — | — | — | — | — | — | — | — | — | |
| MaxRényi-K%Target Model=LLaVA, α=1, K=0%2024.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.572 | — | — | — | — | — | — | — | — | |
| MaxRényi-K%Target Model=LLaVA, α=1, K=10%2024.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.804 | — | — | — | — | — | — | — | |
| MaxRényi-K%Target Model=LLaVA, α=1, K=100%2024.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.762 | — | — | — | — | — | — | |
| MaxRényi-K%Target Model=LLaVA, α=2, K=0%2024.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.608 | — | — | — | — | — | |
| MaxRényi-K%Target Model=LLaVA, α=2, K=10%2024.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.77 | — | — | — | — | |
| MaxRényi-K%Target Model=LLaVA, α=2, K=100%2024.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.72 | — | — | — | |
| MaxRényi-K%Target Model=LLaVA, α=∞, K=0%2024.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.609 | — | — | |
| MaxRényi-K%Target Model=LLaVA, α=∞, K=10%2024.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.742 | — | |
| MaxRényi-K%Target Model=LLaVA, α=∞, K=100%2024.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.703 | |
| Min-K% ProbTarget Model=LLaVA, K=0%2024.11 | — | — | — | — | 0.527 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Min-K% ProbTarget Model=LLaVA, K=10%2024.11 | — | — | — | — | — | 0.908 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Min-K% ProbTarget Model=LLaVA, K=20%2024.11 | — | — | — | — | — | — | 0.982 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ModRényiTarget Model=LLaVA2024.11 | — | — | — | — | — | — | — | — | 0.979 | 0.993 | 0.963 | — | — | — | — | — | — | — | — | — | — | — | |
| ModRényiTarget Model=MiniGPT-42024.11 | — | — | — | — | — | — | — | — | 0.986 | 0.994 | 0.978 | — | — | — | — | — | — | — | — | — | — | — | |
| ModRényiTarget Model=LLaMA Adapter2024.11 | — | — | — | — | — | — | — | — | 0.987 | 0.996 | 0.976 | — | — | — | — | — | — | — | — | — | — | — | |
| ModRényiTarget Model=LLaVA, α=0.52024.11 | — | — | — | — | — | — | — | — | 0.977 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ModRényiTarget Model=LLaVA, α=12024.11 | — | — | — | — | — | — | — | — | — | 0.992 | — | — | — | — | — | — | — | — | — | — | — | — | |
| ModRényiTarget Model=LLaVA, α=22024.11 | — | — | — | — | — | — | — | — | — | — | 0.961 | — | — | — | — | — | — | — | — | — | — | — | |
| PerplexityTarget Model=LLaVA2024.11 | — | 0.986 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Perplexity/lowercaseTarget Model=LLaVA2024.11 | — | — | — | 0.983 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Perplexity/zlibTarget Model=LLaVA2024.11 | — | — | 0.988 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |