Target behavior attack on HumanEval
98.3ASRAiTM
Evaluation results
| Method | Links | |
|---|---|---|
| AiTMFramework=MetaGPT, Victim Agent=Architect2025.02 | 98.3 | |
| AiTMFramework=Camel, Communication Structure=Chain2025.02 | 97.6 | |
| AiTMFramework=MetaGPT, Victim Agent=Project Manager2025.02 | 97.6 | |
| AiTMFramework=AutoGen, Communication Structure=Complete2025.02 | 96.3 | |
| AiTMFramework=Camel, Communication Structure=Complete2025.02 | 96.2 | |
| AiTMFramework=AutoGen, Communication Structure=Chain2025.02 | 95.2 | |
| AiTMFramework=Camel, Communication Structure=Tree2025.02 | 94.7 | |
| AiTMFramework=AutoGen, Communication Structure=Tree2025.02 | 90.4 | |
| AiTMFramework=MetaGPT, Victim Agent=Product manager2025.02 | 90.4 | |
| AiTMFramework=AutoGen, Communication Structure=Random2025.02 | 82.6 | |
| AiTMFramework=Camel, Communication Structure=Random2025.02 | 76.5 | |
| AiTMFramework=MetaGPT, Victim Agent=Engineer2025.02 | 75.7 | |
| AiTMFramework=ChatDev, Victim Agent=Programmer2025.02 | 60.1 | |
| AiTMFramework=ChatDev, Victim Agent=CTO2025.02 | 52.7 | |
| AiTMFramework=ChatDev, Victim Agent=CPO2025.02 | 0 | |
| AiTMFramework=ChatDev, Victim Agent=CEO2025.02 | 0 |