List your company
ResearchFormerly WizWand

Data Extraction Attack on RAP

100Attack Success Rate (ASR)RAG-Thief
-3.16823.61650.477.184Apr 10, 2026Apr 17, 2026Apr 24, 2026May 1, 2026May 8, 2026May 15, 2026May 22, 2026
Updated 5mo ago

Evaluation results

MethodLinks
RAG-ThiefModel=Mistral-7B-Instruct, Number of prompts=30, Memory size=3002026.04100384213
RAG-ThiefModel=Qwen2-72B, Number of prompts=30, Memory size=3002026.0410037410
RAG-ThiefModel=ChatGPT-4, Number of prompts=30, Memory size=3002026.0410035390
PirateModel=Qwen2-72B, Number of prompts=30, Memory size=3002026.04100444910
PirateModel=ChatGPT-4, Number of prompts=30, Memory size=3002026.04100434810
ADAMModel=Mistral-7B-Instruct, Number of prompts=30, Memory size=3002026.04100626967
ADAMModel=Qwen2-72B, Number of prompts=30, Memory size=3002026.04100667360
ADAMModel=ChatGPT-4, Number of prompts=30, Memory size=3002026.04100738177
ADAMModel=Llama2-7b-chat, Number of prompts=30, Memory size=3002026.0498576343
PirateModel=Mistral-7B-Instruct, Number of prompts=30, Memory size=3002026.049741467
PirateModel=Llama2-7b-chat, Number of prompts=30, Memory size=3002026.0495374110
RAG-ThiefModel=Llama2-7b-chat, Number of prompts=30, Memory size=3002026.0487323610
MEXTRAModel=Llama2-7b-chat, Number of prompts=30, Memory size=3002026.0463283113
MEXTRAModel=ChatGPT-4, Number of prompts=30, Memory size=3002026.0463333723
MEXTRAModel=Qwen2-72B, Number of prompts=30, Memory size=3002026.0460303320
MEXTRAModel=Mistral-7B-Instruct, Number of prompts=30, Memory size=3002026.0450273020
VanillaModel=ChatGPT-4, Number of prompts=30, Memory size=3002026.044012110
VanillaModel=Llama2-7b-chat, Number of prompts=30, Memory size=3002026.043310110
VanillaModel=Qwen2-72B, Number of prompts=30, Memory size=3002026.04309200
VanillaModel=Mistral-7B-Instruct, Number of prompts=30, Memory size=3002026.0427890
RDModel=GPT-4o, rho (ρ)=0.152026.050.833———
RFModel=GPT-4o, rho (ρ)=0.152026.050.833———
NNCModel=GPT-4o, rho (ρ)=0.152026.050.833———
MemAuditModel=GPT-4o, rho (ρ)=0.152026.050.833———
RDModel=GPT-4o-mini, rho (ρ)=0.152026.050.8———
RFModel=GPT-4o-mini, rho (ρ)=0.152026.050.8———
NNCModel=GPT-4o-mini, rho (ρ)=0.152026.050.8———
MemAuditModel=GPT-4o-mini, rho (ρ)=0.152026.050.8———
RDModel=DeepSeek, rho (ρ)=0.152026.050.8———
RFModel=DeepSeek, rho (ρ)=0.152026.050.8———
NNCModel=DeepSeek, rho (ρ)=0.152026.050.8———
MemAuditModel=DeepSeek, rho (ρ)=0.152026.050.8———