Prompt Leakage Attack on PubMedQA
14ASR (500)OPTILEAK
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| OPTILEAKModel=Llama-3.1-8B-Instruct2026.02 | 14 | 33 | 93 | 94 | 241.48 | |
| SFTModel=Llama-3.1-8B-Instruct2026.02 | 13 | 33 | 93 | 94 | 254.86 | |
| SFTModel=Qwen-2.5-3B-Instruct2026.02 | 12 | 28 | 94 | 91 | 220.85 | |
| OPTILEAKModel=Qwen-2.5-14B-Instruct2026.02 | 12 | 30 | 89 | 88 | 304.76 | |
| OPTILEAKModel=Qwen-2.5-3B-Instruct2026.02 | 11 | 31 | 94 | 91 | 208.72 | |
| SFTModel=Qwen-2.5-7B-Instruct2026.02 | 9 | 21 | 86 | 91 | 291.28 | |
| OPTILEAKModel=Qwen-2.5-7B-Instruct2026.02 | 6 | 22 | 88 | 91 | 274.91 | |
| SFTModel=Qwen-2.5-14B-Instruct2026.02 | 4 | 24 | 81 | 88 | 374.85 | |
| DPOModel=Qwen-2.5-3B-Instruct2026.02 | 1 | 5 | 71 | 63 | 571.51 | |
| BaseModel=Qwen-2.5-3B-Instruct2026.02 | 0 | 3 | 68 | — | 628.5 | |
| BaseModel=Qwen-2.5-7B-Instruct2026.02 | 0 | 1 | 49 | — | 943.05 | |
| DPOModel=Qwen-2.5-7B-Instruct2026.02 | 0 | 1 | 49 | 70 | 901.09 | |
| BaseModel=Qwen-2.5-14B-Instruct2026.02 | 0 | 0 | 58 | — | 796.08 | |
| DPOModel=Qwen-2.5-14B-Instruct2026.02 | 0 | 0 | 59 | 40 | 753.28 | |
| BaseModel=Llama-3.1-8B-Instruct2026.02 | 0 | 0 | 57 | — | 801.71 | |
| DPOModel=Llama-3.1-8B-Instruct2026.02 | 0 | 0 | 60 | 84 | 666.01 |