Healthcare Record Management on EHRAgent
74.8Accuracy (ACC)AGENTPOISON
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| AGENTPOISONAgent Backbone=ChatGPT, Retriever Type=end-to-end-retriever2024.07 | 74.8 | 97.9 | 91.7 | 53.7 | |
| Non-attackAgent Backbone=ChatGPT, Retriever Type=contrastive-retriever2024.07 | 73 | — | — | — | |
| AGENTPOISONAgent Backbone=ChatGPT, Retriever Type=contrastive-retriever2024.07 | 72.9 | 98.9 | 97.9 | 58.3 | |
| BadChainAgent Backbone=ChatGPT, Retriever Type=end-to-end-retriever2024.07 | 71.9 | 21.1 | 50.5 | 33.7 | |
| Non-attackAgent Backbone=ChatGPT, Retriever Type=end-to-end-retriever2024.07 | 71.6 | — | — | — | |
| BadChainAgent Backbone=ChatGPT, Retriever Type=contrastive-retriever2024.07 | 70.8 | 11.2 | 72.5 | 8.3 | |
| GCGAgent Backbone=ChatGPT, Retriever Type=end-to-end-retriever2024.07 | 70.8 | 12.5 | 63.5 | 30.2 | |
| GCGAgent Backbone=ChatGPT, Retriever Type=contrastive-retriever2024.07 | 70.1 | 9.4 | 81.3 | 45.8 | |
| AutoDANAgent Backbone=ChatGPT, Retriever Type=contrastive-retriever2024.07 | 68.4 | 84.2 | 89.5 | 27.4 | |
| CPAAgent Backbone=ChatGPT, Retriever Type=contrastive-retriever2024.07 | 67.9 | 96.9 | 58.3 | 51.1 | |
| AutoDANAgent Backbone=ChatGPT, Retriever Type=end-to-end-retriever2024.07 | 67.4 | 38.9 | 51.6 | 42.1 | |
| CPAAgent Backbone=ChatGPT, Retriever Type=end-to-end-retriever2024.07 | 66.3 | 61.5 | 55.8 | 38.5 | |
| GCGAgent Backbone=LLaMA3, Retriever Type=contrastive-retriever2024.07 | 44.2 | 16.4 | 14.8 | 29.5 | |
| AGENTPOISONAgent Backbone=LLaMA3, Retriever Type=contrastive-retriever2024.07 | 41 | 100 | 21.5 | 65.6 | |
| Non-attackAgent Backbone=LLaMA3, Retriever Type=contrastive-retriever2024.07 | 37.7 | — | — | — | |
| AutoDANAgent Backbone=LLaMA3, Retriever Type=contrastive-retriever2024.07 | 36.1 | 75.4 | 6.6 | 57.4 | |
| CPAAgent Backbone=LLaMA3, Retriever Type=contrastive-retriever2024.07 | 34.4 | 96.9 | 24.6 | 72.1 | |
| GCGAgent Backbone=LLaMA3, Retriever Type=end-to-end-retriever2024.07 | 34.4 | 8.8 | 11.5 | 19.7 | |
| AGENTPOISONAgent Backbone=LLaMA3, Retriever Type=end-to-end-retriever2024.07 | 34.4 | 96.7 | 7.7 | 68.9 | |
| Non-attackAgent Backbone=LLaMA3, Retriever Type=end-to-end-retriever2024.07 | 32.8 | — | — | — | |
| AutoDANAgent Backbone=LLaMA3, Retriever Type=end-to-end-retriever2024.07 | 31.1 | 13.1 | 1.6 | 8.2 | |
| BadChainAgent Backbone=LLaMA3, Retriever Type=end-to-end-retriever2024.07 | 31.1 | 6.2 | 8.2 | 13.1 | |
| BadChainAgent Backbone=LLaMA3, Retriever Type=contrastive-retriever2024.07 | 29.5 | 31.1 | 18 | 65.6 | |
| CPAAgent Backbone=LLaMA3, Retriever Type=end-to-end-retriever2024.07 | 21.3 | 15.3 | 4.8 | 8.6 |