Harmful Refusal on DAN
94.9ASRQwen2.5-3B-IT
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen2.5-3B-ITBackbone=Qwen2.5-3B-IT2026.05 | 94.9 | |
| Llama3.1-8B-IT-ABTraining=Instruction-tuned ablated base2025.06 | 93.7 | |
| Qwen2.5-7B-ITBackbone=Qwen2.5-7B-IT2026.05 | 89.7 | |
| SELF-REDTEAMBackbone=Qwen2.5-3B-IT2026.05 | 88.5 | |
| Qwen2.5-14B-ITBackbone=Qwen2.5-14B-IT2026.05 | 84.5 | |
| ABSBackbone=Qwen2.5-7B-IT2026.05 | 75.6 | |
| SELF-REDTEAMBackbone=Qwen2.5-7B-IT2026.05 | 72.6 | |
| ABSBackbone=Qwen2.5-3B-IT2026.05 | 72.1 | |
| ABSBackbone=Qwen2.5-14B-IT2026.05 | 70.6 | |
| SELF-REDTEAMBackbone=Qwen2.5-14B-IT2026.05 | 66.1 | |
| Defender-OnlyBase Model=Llama-3.1-8B-IT, Input Condition=adv harm2025.10 | 54.2 | |
| Llama3.1-8B-ITTraining=Instruction-tuned base2025.06 | 54 | |
| Self-PlayBase Model=Llama-3.1-8B-IT, Input Condition=adv harm2025.10 | 53.7 | |
| Llama-3.1-8B-ITModel Status=Base Model, Input Condition=adv harm2025.10 | 53.3 | |
| Qwen2.5-3B-ITTraining=Instruction-tuned base2025.06 | 51.7 | |
| SFTBase Model=Llama-3.1-8B-IT, Input Condition=adv harm2025.10 | 46.8 | |
| Defender-Only + SFTBase Model=Llama-3.1-8B-IT, Input Condition=adv harm2025.10 | 45.2 | |
| Self-Play + SFTBase Model=Llama-3.1-8B-IT, Input Condition=adv harm2025.10 | 39.6 | |
| Llama3.1-8B-IT-AB + SELF-REDTEAMTraining=SELF-REDTEAM fine-tuned2025.06 | 39.6 | |
| Qwen2.5-7B-ITTraining=Instruction-tuned base2025.06 | 39 | |
| ELSBase Model=Llama-3.1-8B-IT, Input Condition=adv harm2025.10 | 37.2 | |
| Qwen2.5-3B-IT + SELF-REDTEAMTraining=SELF-REDTEAM fine-tuned2025.06 | 33 | |
| Llama3.1-8B-IT + SELF-REDTEAMTraining=SELF-REDTEAM fine-tuned2025.06 | 23.9 | |
| Qwen2.5-7B-IT + SELF-REDTEAMTraining=SELF-REDTEAM fine-tuned2025.06 | 22.2 | |
| Qwen2.5-14B-ITTraining=Instruction-tuned base2025.06 | 21.7 | |
| Qwen2.5-14B-IT + SELF-REDTEAMTraining=SELF-REDTEAM fine-tuned2025.06 | 10.6 |