Data Extraction Attack on ReAct
86EQADAM
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| ADAMModel=ChatGPT-4, Number of prompts=30, Memory size=3002026.04 | 86 | 95 | 90 | 100 | |
| ADAMModel=Qwen2-72B, Number of prompts=30, Memory size=3002026.04 | 79 | 88 | 80 | 100 | |
| ADAMModel=Llama2-7b-chat, Number of prompts=30, Memory size=3002026.04 | 75 | 83 | 76 | 100 | |
| ADAMModel=Mistral-7B-Instruct, Number of prompts=30, Memory size=3002026.04 | 70 | 78 | 70 | 100 | |
| PirateModel=Mistral-7B-Instruct, Number of prompts=30, Memory size=3002026.04 | 50 | 56 | 13 | 100 | |
| PirateModel=Qwen2-72B, Number of prompts=30, Memory size=3002026.04 | 48 | 53 | 20 | 100 | |
| PirateModel=ChatGPT-4, Number of prompts=30, Memory size=3002026.04 | 47 | 52 | 15 | 100 | |
| PirateModel=Llama2-7b-chat, Number of prompts=30, Memory size=3002026.04 | 46 | 51 | 25 | 90 | |
| MEXTRAModel=ChatGPT-4, Number of prompts=30, Memory size=3002026.04 | 41 | 45 | 23 | 57 | |
| MEXTRAModel=Qwen2-72B, Number of prompts=30, Memory size=3002026.04 | 38 | 42 | 20 | 67 | |
| MEXTRAModel=Llama2-7b-chat, Number of prompts=30, Memory size=3002026.04 | 36 | 40 | 16 | 47 | |
| RAG-ThiefModel=Mistral-7B-Instruct, Number of prompts=30, Memory size=3002026.04 | 33 | 37 | 10 | 90 | |
| RAG-ThiefModel=ChatGPT-4, Number of prompts=30, Memory size=3002026.04 | 33 | 37 | 10 | 90 | |
| MEXTRAModel=Mistral-7B-Instruct, Number of prompts=30, Memory size=3002026.04 | 33 | 36 | 17 | 36 | |
| RAG-ThiefModel=Qwen2-72B, Number of prompts=30, Memory size=3002026.04 | 30 | 33 | 7 | 86 | |
| RAG-ThiefModel=Llama2-7b-chat, Number of prompts=30, Memory size=3002026.04 | 28 | 31 | 0 | 93 | |
| VanillaModel=Qwen2-72B, Number of prompts=30, Memory size=3002026.04 | 13 | 14 | 0 | 43 | |
| VanillaModel=ChatGPT-4, Number of prompts=30, Memory size=3002026.04 | 13 | 14 | 0 | 43 | |
| VanillaModel=Llama2-7b-chat, Number of prompts=30, Memory size=3002026.04 | 12 | 13 | 0 | 40 | |
| VanillaModel=Mistral-7B-Instruct, Number of prompts=30, Memory size=3002026.04 | 11 | 12 | 0 | 37 |