Privacy Attack Evaluation on Travel Planning
88.51ASRGPT-5
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GPT-5Firewall configuration=w/o Firewall2025.02 | 88.51 | 8.07 | 96.58 | |
| Gemini 2.5 ProFirewall configuration=w/o Firewall2025.02 | 67.16 | 7.77 | 90.5 | |
| Claude Sonnet 4Firewall configuration=w/o Firewall2025.02 | 55.77 | 8.12 | 95.17 | |
| GPT-5Firewall protection=w/o Firewall2025.02 | 55.32 | 7.71 | 95.44 | |
| Gemini 2.5 ProFirewall protection=w/o Firewall2025.02 | 32.58 | 7.81 | 90.29 | |
| Gemini 2.5 FlashFirewall configuration=w/o Firewall2025.02 | 27.56 | 7.19 | 83.12 | |
| Gemini 2.5 FlashFirewall protection=w/o Firewall2025.02 | 18.95 | 7.38 | 78.87 | |
| Gemini 2.5 ProFirewall configuration=w/ Firewall2025.02 | 9.18 | 7.73 | 86.65 | |
| Gemini 2.5 FlashFirewall configuration=w/ Firewall2025.02 | 8.08 | 7.93 | 96.24 | |
| GPT-5Firewall configuration=w/ Firewall2025.02 | 7.77 | 8.42 | 95.63 | |
| Claude Sonnet 4Firewall configuration=w/ Firewall2025.02 | 7.25 | 8.45 | 98.21 | |
| Claude Sonnet 4Firewall protection=w/o Firewall2025.02 | 4.35 | 8.1 | 94.25 | |
| GPT-5Firewall protection=w/ Firewall2025.02 | 3.26 | 8.27 | 97 | |
| Gemini 2.5 ProFirewall protection=w/ Firewall2025.02 | 2.33 | 8.08 | 86.88 | |
| Gemini 2.5 FlashFirewall protection=w/ Firewall2025.02 | 1.15 | 7.84 | 91.13 | |
| Claude Sonnet 4Firewall protection=w/ Firewall2025.02 | 1.1 | 8.3 | 98.12 |