Multi-agent System Safety Evaluation on PsySafe Dangerous Tasks 1.0
21.5JDR R3w/o Defense
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| w/o DefenseDefense Strategy=None2024.01 | 21.5 | 27.6 | 38.4 | 98.4 | — | — | |
| Llama GuardDefense Type=Content Filtering2024.01 | 13.8 | 18.4 | 26.2 | 49.2 | — | 49.2 | |
| Police DefenseDefense Mechanism=Police agent criticism2024.01 | 4.6 | 10.7 | 38.4 | 98.4 | — | — | |
| Doctor DefenseDefense Mechanism=Psychological state treatment2024.01 | 1.5 | 1.5 | 3 | 26.2 | — | — | |
| GPT-4Model Version=GPT-4 0613, Bypass Mode=False2024.01 | 0 | 0 | 1.5 | 4.6 | — | 95 | |
| GPT-4†Model Version=GPT-4 0613, Bypass Mode=True (simple jailbreak prompt)2024.01 | 0 | 26.2 | 24.6 | 29.2 | — | 6.2 |