Data Extraction Attack on RAP
100Attack Success Rate (ASR)RAG-Thief
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| RAG-ThiefModel=Mistral-7B-Instruct, Number of prompts=30, Memory size=3002026.04 | 100 | 38 | 42 | 13 | |
| RAG-ThiefModel=Qwen2-72B, Number of prompts=30, Memory size=3002026.04 | 100 | 37 | 41 | 0 | |
| RAG-ThiefModel=ChatGPT-4, Number of prompts=30, Memory size=3002026.04 | 100 | 35 | 39 | 0 | |
| PirateModel=Qwen2-72B, Number of prompts=30, Memory size=3002026.04 | 100 | 44 | 49 | 10 | |
| PirateModel=ChatGPT-4, Number of prompts=30, Memory size=3002026.04 | 100 | 43 | 48 | 10 | |
| ADAMModel=Mistral-7B-Instruct, Number of prompts=30, Memory size=3002026.04 | 100 | 62 | 69 | 67 | |
| ADAMModel=Qwen2-72B, Number of prompts=30, Memory size=3002026.04 | 100 | 66 | 73 | 60 | |
| ADAMModel=ChatGPT-4, Number of prompts=30, Memory size=3002026.04 | 100 | 73 | 81 | 77 | |
| ADAMModel=Llama2-7b-chat, Number of prompts=30, Memory size=3002026.04 | 98 | 57 | 63 | 43 | |
| PirateModel=Mistral-7B-Instruct, Number of prompts=30, Memory size=3002026.04 | 97 | 41 | 46 | 7 | |
| PirateModel=Llama2-7b-chat, Number of prompts=30, Memory size=3002026.04 | 95 | 37 | 41 | 10 | |
| RAG-ThiefModel=Llama2-7b-chat, Number of prompts=30, Memory size=3002026.04 | 87 | 32 | 36 | 10 | |
| MEXTRAModel=Llama2-7b-chat, Number of prompts=30, Memory size=3002026.04 | 63 | 28 | 31 | 13 | |
| MEXTRAModel=ChatGPT-4, Number of prompts=30, Memory size=3002026.04 | 63 | 33 | 37 | 23 | |
| MEXTRAModel=Qwen2-72B, Number of prompts=30, Memory size=3002026.04 | 60 | 30 | 33 | 20 | |
| MEXTRAModel=Mistral-7B-Instruct, Number of prompts=30, Memory size=3002026.04 | 50 | 27 | 30 | 20 | |
| VanillaModel=ChatGPT-4, Number of prompts=30, Memory size=3002026.04 | 40 | 12 | 11 | 0 | |
| VanillaModel=Llama2-7b-chat, Number of prompts=30, Memory size=3002026.04 | 33 | 10 | 11 | 0 | |
| VanillaModel=Qwen2-72B, Number of prompts=30, Memory size=3002026.04 | 30 | 9 | 20 | 0 | |
| VanillaModel=Mistral-7B-Instruct, Number of prompts=30, Memory size=3002026.04 | 27 | 8 | 9 | 0 | |
| RDModel=GPT-4o, rho (ρ)=0.152026.05 | 0.833 | — | — | — | |
| RFModel=GPT-4o, rho (ρ)=0.152026.05 | 0.833 | — | — | — | |
| NNCModel=GPT-4o, rho (ρ)=0.152026.05 | 0.833 | — | — | — | |
| MemAuditModel=GPT-4o, rho (ρ)=0.152026.05 | 0.833 | — | — | — | |
| RDModel=GPT-4o-mini, rho (ρ)=0.152026.05 | 0.8 | — | — | — | |
| RFModel=GPT-4o-mini, rho (ρ)=0.152026.05 | 0.8 | — | — | — | |
| NNCModel=GPT-4o-mini, rho (ρ)=0.152026.05 | 0.8 | — | — | — | |
| MemAuditModel=GPT-4o-mini, rho (ρ)=0.152026.05 | 0.8 | — | — | — | |
| RDModel=DeepSeek, rho (ρ)=0.152026.05 | 0.8 | — | — | — | |
| RFModel=DeepSeek, rho (ρ)=0.152026.05 | 0.8 | — | — | — | |
| NNCModel=DeepSeek, rho (ρ)=0.152026.05 | 0.8 | — | — | — | |
| MemAuditModel=DeepSeek, rho (ρ)=0.152026.05 | 0.8 | — | — | — |