Safety Evaluation on BBQ
9.21Harmful Content Rate (HCRate)DeepSeek-V3
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DeepSeek-V3Attack=None2026.06 | 9.21 | 78 | |
| GPT-4oAttack=None2026.06 | 13.16 | 59.27 | |
| Claude-3.7-SonnetAttack=None2026.06 | 13.16 | 43.82 | |
| GHOSTWRITERTarget Model=GPT-4o, Attacker LM=GPT-4o-mini2026.06 | 76.32 | 81.64 | |
| GHOSTWRITERTarget Model=Claude-3.7-Sonnet, Attacker LM=Qwen-2.5-7B2026.06 | 102.63 | 73.45 | |
| GHOSTWRITERTarget Model=Claude-3.7-Sonnet, Attacker LM=GPT-4o-mini2026.06 | 103.95 | 87.82 | |
| GHOSTWRITERTarget Model=GPT-4o, Attacker LM=Qwen-2.5-7B2026.06 | 139.47 | 83.45 | |
| GHOSTWRITERTarget Model=DeepSeek-V3, Attacker LM=GPT-4o-mini2026.06 | 176.32 | 96 | |
| GHOSTWRITERTarget Model=DeepSeek-V3, Attacker LM=Qwen-2.5-7B2026.06 | 269.74 | 97.64 |