Membership Inference Attack on VL-MIA Flickr 1.0 (test)
8TPR@5%FPR (Aug_KL)MiniGPT-4
Evaluation results
| Method | Links | |||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MiniGPT-4Logits Slice=img2024.11 | 8 | — | — | — | — | 17.7 | — | — | — | 3.3 | 5 | 18.7 | 2.7 | 7 | 8.3 | 5.3 | 5 | 12 | 5 | 5 | 12.3 | |
| LLaVALogits Slice=desp2024.11 | 5.3 | 13 | 9.3 | 8.3 | 12.7 | 16 | 11.7 | 11.7 | 11.7 | 8.3 | 6.3 | 16.3 | 12 | 7 | 13.7 | 9.3 | 9.3 | 17.3 | 9.3 | 8.3 | 13 | |
| LLaVALogits Slice=inst+desp2024.11 | 4.3 | 7 | 2.3 | 1.3 | 0.3 | 16 | 11 | 7 | 11.7 | 20 | 14 | 17.7 | 16.7 | 12 | 16 | 15 | 10.3 | 19.7 | 17 | 11.3 | 16.7 | |
| LLaVALogits Slice=inst2024.11 | 3.3 | 0.7 | 2.3 | 2.3 | 0.7 | 8.3 | 0.3 | 0.7 | 0.3 | 20.3 | 20.3 | 21.7 | 13.3 | 13.3 | 11.7 | 11.3 | 11.3 | 9.3 | 9.7 | 9.7 | 11.3 | |
| LLaVALogits Slice=img2024.11 | 2.7 | — | — | — | — | 5.3 | — | — | — | 4.7 | 11 | 10.3 | 5.3 | 8.7 | 9 | 7 | 8.3 | 9 | 9.7 | 12.7 | 8.7 | |
| MiniGPT-4Logits Slice=desp2024.11 | 1.7 | 6.7 | 8.3 | 10.3 | 11.3 | 2.3 | 8 | 8 | 8 | 13.3 | 13 | 9 | 11 | 13 | 8.7 | 9 | 10.3 | 7.3 | 8.3 | 10.3 | 6.7 | |
| MiniGPT-4Logits Slice=inst2024.11 | 1.3 | 0 | 1 | 1 | 0.3 | 11.3 | 4.3 | 0.3 | 10 | 5.3 | 5.3 | 18 | 5 | 5 | 15.7 | 16.7 | 16.7 | 20 | 16.3 | 16.3 | 17 | |
| MiniGPT-4Logits Slice=inst+desp2024.11 | 1.3 | 7.3 | 0.7 | 0.3 | 0.3 | 3.3 | 11 | 1.3 | 10.7 | 5.3 | 11.7 | 22.3 | 7.3 | 13 | 17.3 | 18.3 | 22 | 17.7 | 16.3 | 11 | 15 | |
| LLaMA AdapterLogits Slice=inst2024.11 | 1 | 1.7 | 3.3 | 2 | 1.7 | 3.7 | 0 | 1.3 | 0 | 6.3 | 4 | 8.7 | 7.7 | 4.3 | 8 | 9.7 | 10.3 | 9 | 13.3 | 11.3 | 7.3 | |
| LLaMA AdapterLogits Slice=desp2024.11 | 1 | 15.7 | 12 | 9.7 | 11 | 14.3 | 14.7 | 15 | 15.7 | 5.3 | 6.7 | 15.7 | 4.7 | 5.7 | 19.7 | 6 | 7.3 | 19.7 | 12 | 9.7 | 15.7 | |
| LLaMA AdapterLogits Slice=inst+desp2024.11 | 0.7 | 2.3 | 3.3 | 1.7 | 1.7 | 12.7 | 4 | 3.7 | 4.7 | 6.3 | 10 | 24.7 | 8 | 8 | 20.3 | 10.7 | 15.7 | 14 | 14 | 11.7 | 19 |