Knowledge-intensive QA on StrategyQA
66.7ACCNon-attack
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Non-attackAgent Backbone=ChatGPT, Retriever Type=contrastive-retriever2024.07 | 66.7 | — | — | — | |
| AGENTPOISONAgent Backbone=ChatGPT, Retriever Type=contrastive-retriever2024.07 | 65.7 | 65.5 | 73.6 | 58.6 | |
| Non-attackAgent Backbone=ChatGPT, Retriever Type=end-to-end-retriever2024.07 | 59.6 | — | — | — | |
| BadChainAgent Backbone=ChatGPT, Retriever Type=end-to-end-retriever2024.07 | 58.6 | 2.8 | 33.3 | 44.4 | |
| AGENTPOISONAgent Backbone=ChatGPT, Retriever Type=end-to-end-retriever2024.07 | 57.6 | 64.7 | 54.7 | 70.7 | |
| CPAAgent Backbone=ChatGPT, Retriever Type=end-to-end-retriever2024.07 | 57.1 | 22.2 | 50 | 51.6 | |
| GCGAgent Backbone=ChatGPT, Retriever Type=contrastive-retriever2024.07 | 56.6 | 40.2 | 30.8 | 38.4 | |
| AutoDANAgent Backbone=ChatGPT, Retriever Type=end-to-end-retriever2024.07 | 56.1 | 17.6 | 48.5 | 48.5 | |
| CPAAgent Backbone=ChatGPT, Retriever Type=contrastive-retriever2024.07 | 55.6 | 52.8 | 66.7 | 48.9 | |
| GCGAgent Backbone=ChatGPT, Retriever Type=end-to-end-retriever2024.07 | 54.5 | 19.5 | 30.8 | 49.5 | |
| AutoDANAgent Backbone=ChatGPT, Retriever Type=contrastive-retriever2024.07 | 51.6 | 42.9 | 28.3 | 49.5 | |
| Non-attackAgent Backbone=LLaMA3, Retriever Type=end-to-end-retriever2024.07 | 51 | — | — | — | |
| BadChainAgent Backbone=ChatGPT, Retriever Type=contrastive-retriever2024.07 | 50.5 | 49.4 | 65.2 | 52.9 | |
| BadChainAgent Backbone=LLaMA3, Retriever Type=end-to-end-retriever2024.07 | 49 | 1.2 | 0 | 45.1 | |
| Non-attackAgent Backbone=LLaMA3, Retriever Type=contrastive-retriever2024.07 | 47.5 | — | — | — | |
| AGENTPOISONAgent Backbone=LLaMA3, Retriever Type=contrastive-retriever2024.07 | 47.5 | 58.4 | 22.5 | 72.3 | |
| CPAAgent Backbone=LLaMA3, Retriever Type=end-to-end-retriever2024.07 | 47.5 | 28.1 | 30 | 52.9 | |
| AGENTPOISONAgent Backbone=LLaMA3, Retriever Type=end-to-end-retriever2024.07 | 47 | 66.7 | 21.7 | 72.5 | |
| GCGAgent Backbone=LLaMA3, Retriever Type=contrastive-retriever2024.07 | 45.6 | 36.7 | 29.6 | 64.4 | |
| BadChainAgent Backbone=LLaMA3, Retriever Type=contrastive-retriever2024.07 | 39.6 | 44.6 | 23.1 | 62.4 | |
| GCGAgent Backbone=LLaMA3, Retriever Type=end-to-end-retriever2024.07 | 37.3 | 19.1 | 25 | 37.3 | |
| AutoDANAgent Backbone=LLaMA3, Retriever Type=end-to-end-retriever2024.07 | 37.3 | 11 | 34.1 | 22.7 | |
| CPAAgent Backbone=LLaMA3, Retriever Type=contrastive-retriever2024.07 | 37 | 52 | 25 | 58.5 | |
| AutoDANAgent Backbone=LLaMA3, Retriever Type=contrastive-retriever2024.07 | 36.6 | 48.5 | 41.3 | 68.3 |