Membership Inference Attack on SCIDOCS
0.991AUCMEntA
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MEntALLM Generator=Phi4-14B, Query Budget=5, RAG Setting=Fixed2026.05 | 0.991 | 96.7 | 90.6 | 96.9 | |
| MEntALLM Generator=CommandR-7B, Query Budget=5, RAG Setting=Fixed2026.05 | 0.984 | 96.2 | 88.3 | 95.6 | |
| DCMILLM Generator=Phi4-14B, RAG Setting=Fixed2026.05 | 0.941 | 94 | — | — | |
| MEntALLM Generator=Llama3.1-8B, Query Budget=5, RAG Setting=Fixed2026.05 | 0.936 | 91.1 | 62.3 | 79.5 | |
| IALLM Generator=Llama3.1-8B, Query Budget=5, RAG Setting=Fixed2026.05 | 0.93 | 86.5 | 0 | 75.1 | |
| IALLM Generator=Phi4-14B, Query Budget=5, RAG Setting=Fixed2026.05 | 0.915 | 83.3 | 0 | 62.8 | |
| IALLM Generator=CommandR-7B, Query Budget=5, RAG Setting=Fixed2026.05 | 0.91 | 84.1 | 0 | 63.4 | |
| MBALLM Generator=Phi4-14B, RAG Setting=Fixed2026.05 | 0.891 | 87.4 | 73.2 | 79.7 | |
| DCMILLM Generator=Llama3.1-8B, RAG Setting=Fixed2026.05 | 0.857 | 85.4 | — | — | |
| MBALLM Generator=Llama3.1-8B, RAG Setting=Fixed2026.05 | 0.802 | 68.9 | 24.5 | 53.9 | |
| MEntALLM Generator=Gemma2-2B, Query Budget=5, RAG Setting=Fixed2026.05 | 0.799 | 78.1 | 31 | 53.6 | |
| DCMILLM Generator=CommandR-7B, RAG Setting=Fixed2026.05 | 0.788 | 78.6 | — | — | |
| S2MIALLM Generator=Gemma2-2B, RAG Setting=Fixed2026.05 | 0.726 | 69.1 | 11.8 | 26.6 | |
| S2MIALLM Generator=Phi4-14B, RAG Setting=Fixed2026.05 | 0.715 | 67.4 | 19.7 | 32.2 | |
| IALLM Generator=Gemma2-2B, Query Budget=5, RAG Setting=Fixed2026.05 | 0.701 | 66.3 | 10.4 | 10.4 | |
| MBALLM Generator=CommandR-7B, RAG Setting=Fixed2026.05 | 0.666 | 59.2 | 25.7 | 35.5 | |
| DCMILLM Generator=Gemma2-2B, RAG Setting=Fixed2026.05 | 0.577 | 56.3 | — | — | |
| S2MIALLM Generator=Llama3.1-8B, RAG Setting=Fixed2026.05 | 0.547 | 53.5 | 5.6 | 10.4 | |
| S2MIALLM Generator=CommandR-7B, RAG Setting=Fixed2026.05 | 0.546 | 56 | 7.9 | 15.1 | |
| MBALLM Generator=Gemma2-2B, RAG Setting=Fixed2026.05 | 0.541 | 50 | 2.1 | 9.6 |