Membership Inference Attack on VLLM instruction-tuning texts length 32 1.0 (test)
0.809ModRényi Divergence (α=0.5)ModRényi
Evaluation results
| Method | Links | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ModRényiTarget Model=LLaVA2024.11 | 0.809 | — | — | — | — | — | — | — | 0.808 | 0.779 | — | — | — | — | — | — | — | — | — | — | — | — | |
| ModRényiTarget Model=LLaVA, α=0.52024.11 | 0.809 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ModRényiTarget Model=LLaMA Adapter2024.11 | 0.844 | — | — | — | — | — | — | — | 0.876 | 0.815 | — | — | — | — | — | — | — | — | — | — | — | — | |
| ModRényiTarget Model=MiniGPT-42024.11 | 0.915 | — | — | — | — | — | — | — | 0.952 | 0.892 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Max_Prob_GapTarget Model=LLaVA2024.11 | — | — | — | — | — | — | — | 0.461 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Max_Prob_GapTarget Model=LLaVA2024.11 | — | — | — | — | — | — | — | 0.465 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MaxRényi-K%Target Model=LLaVA2024.11 | — | — | — | — | — | — | — | — | — | — | 0.506 | 0.458 | 0.564 | 0.552 | 0.566 | 0.554 | 0.589 | 0.607 | 0.553 | 0.6 | 0.618 | 0.557 | |
| MaxRényi-K%Target Model=MiniGPT-42024.11 | — | — | — | — | — | — | — | — | — | — | 0.533 | 0.368 | 0.805 | 0.528 | 0.566 | 0.756 | 0.648 | 0.681 | 0.755 | 0.679 | 0.688 | 0.747 | |
| MaxRényi-K%Target Model=LLaMA Adapter2024.11 | — | — | — | — | — | — | — | — | — | — | 0.517 | 0.505 | 0.57 | 0.59 | 0.591 | 0.561 | 0.615 | 0.61 | 0.554 | 0.612 | 0.615 | 0.549 | |
| MaxRényi-K%Target Model=LLaVA, α=0.5, K=0%2024.11 | — | — | — | — | — | — | — | — | — | — | 0.504 | — | — | — | — | — | — | — | — | — | — | — | |
| MaxRényi-K%Target Model=LLaVA, α=0.5, K=10%2024.11 | — | — | — | — | — | — | — | — | — | — | — | 0.458 | — | — | — | — | — | — | — | — | — | — | |
| MaxRényi-K%Target Model=LLaVA, α=0.5, K=100%2024.11 | — | — | — | — | — | — | — | — | — | — | — | — | 0.573 | — | — | — | — | — | — | — | — | — | |
| MaxRényi-K%Target Model=LLaVA, α=1, K=0%2024.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.544 | — | — | — | — | — | — | — | — | |
| MaxRényi-K%Target Model=LLaVA, α=1, K=10%2024.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.557 | — | — | — | — | — | — | — | |
| MaxRényi-K%Target Model=LLaVA, α=1, K=100%2024.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.555 | — | — | — | — | — | — | |
| MaxRényi-K%Target Model=LLaVA, α=2, K=0%2024.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.577 | — | — | — | — | — | |
| MaxRényi-K%Target Model=LLaVA, α=2, K=10%2024.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.593 | — | — | — | — | |
| MaxRényi-K%Target Model=LLaVA, α=2, K=100%2024.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.554 | — | — | — | |
| MaxRényi-K%Target Model=LLaVA, α=∞, K=0%2024.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.582 | — | — | |
| MaxRényi-K%Target Model=LLaVA, α=∞, K=10%2024.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.595 | — | |
| MaxRényi-K%Target Model=LLaVA, α=∞, K=100%2024.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.555 | |
| Min-K% ProbTarget Model=LLaVA, K=0%2024.11 | — | — | — | — | 0.525 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Min-K% ProbTarget Model=LLaVA, K=10%2024.11 | — | — | — | — | — | 0.463 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Min-K% ProbTarget Model=LLaVA, K=20%2024.11 | — | — | — | — | — | — | 0.598 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Min-k% Prob methodsTarget Model=LLaVA2024.11 | — | — | — | — | 0.522 | 0.461 | 0.603 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ModRényiTarget Model=LLaVA, α=12024.11 | — | — | — | — | — | — | — | — | 0.811 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ModRényiTarget Model=LLaVA, α=22024.11 | — | — | — | — | — | — | — | — | — | 0.78 | — | — | — | — | — | — | — | — | — | — | — | — | |
| PerplexityTarget Model=LLaVA2024.11 | — | 0.784 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Perplexity-based methodsTarget Model=LLaVA2024.11 | — | 0.779 | 0.609 | 0.962 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Perplexity/lowercaseTarget Model=LLaVA2024.11 | — | — | — | 0.965 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Perplexity/zlibTarget Model=LLaVA2024.11 | — | — | 0.626 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |