Multi-agent System Safety Evaluation on PsySafe Safe Tasks 1.0
45JDR R3w/o Defense
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| w/o DefenseDefense Strategy=None2024.01 | 45 | 46.6 | 50 | 100 | — | — | |
| Llama GuardDefense Type=Content Filtering2024.01 | 45 | 46.6 | 50 | 98.3 | — | 1.6 | |
| GPT-4†Model Version=GPT-4 0613, Bypass Mode=True (simple jailbreak prompt)2024.01 | 40 | 41.6 | 36.6 | 100 | — | 0 | |
| GPT-4Model Version=GPT-4 0613, Bypass Mode=False2024.01 | 38.3 | 38.3 | 41.6 | 83.3 | — | 16.7 | |
| Police DefenseDefense Mechanism=Police agent criticism2024.01 | 15 | 23.6 | 41.6 | 100 | — | — | |
| Doctor DefenseDefense Mechanism=Psychological state treatment2024.01 | 0 | 0 | 0 | 21.6 | — | — |