Safety Evaluation on ToxiGen
100SafetyVCL
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| VCLBase Model=Llama-3.1-8B2026.03 | 100 | — | — | |
| Vector AblationBase Model=Llama-3.1-8B2026.03 | 98 | — | — | |
| System PromptBase Model=Llama-3.1-8B2026.03 | 96 | — | — | |
| Llama-3.1-8B-SFTBase Model=Llama-3.1-8B2026.03 | 94 | — | — | |
| DROBase Model=Llama-3.1-8B2026.03 | 94 | — | — | |
| Self-Improving PretrainingPre-training Data=RedPajama, Pre-training Strategy=Self-Improving2026.01 | 93.1 | — | — | |
| Self-CDBase Model=Llama-3.1-8B2026.03 | 93 | — | — | |
| SFT2025.12 | 86.7 | — | — | |
| Llama Pretrain BaselinePre-training Data=RedPajama, Pre-training Strategy=Standard next token prediction2026.01 | 82 | — | — | |
| Llama BasePre-training Strategy=Standard next token prediction2026.01 | 80.1 | — | — | |
| LLaMA2-7B2025.12 | 79.1 | — | — | |
| Contrastive Decoding2025.12 | 73.5 | — | — | |
| Task Vector2025.12 | 73.1 | — | — | |
| Safe Activation2025.12 | 71.9 | — | — | |
| MERGEPRINTalpha=0.1, merging_method=TIES-merging, DARE_augmented=true2024.10 | 54 | — | — | |
| Feature Subtraction2025.12 | 53.5 | — | — | |
| TRAPalpha=0.1, merging_method=TIES-merging, DARE_augmented=true2024.10 | 53 | — | — | |
| MERGEPRINTalpha=0.3, merging_method=TIES-merging, DARE_augmented=true2024.10 | 53 | — | — | |
| TRAPalpha=0.3, merging_method=TIES-merging, DARE_augmented=true2024.10 | 52 | — | — | |
| TRAPalpha=0.5, merging_method=TIES-merging, DARE_augmented=true2024.10 | 52 | — | — | |
| MERGEPRINTalpha=0.5, merging_method=TIES-merging, DARE_augmented=true2024.10 | 52 | — | — | |
| TRAPalpha=0.1, merging_method=TIES-merging, DARE_augmented=false2024.10 | 51 | — | — | |
| MERGEPRINTalpha=0.1, merging_method=TIES-merging, DARE_augmented=false2024.10 | 51 | — | — | |
| TRAPalpha=0.1, merging_method=Task Arithmetic, DARE_augmented=false2024.10 | 50 | — | — | |
| TRAPalpha=0.1, merging_method=Task Arithmetic, DARE_augmented=true2024.10 | 50 | — | — | |
| TRAPalpha=0.3, merging_method=TIES-merging, DARE_augmented=false2024.10 | 50 | — | — | |
| TRAPalpha=0.7, merging_method=TIES-merging, DARE_augmented=true2024.10 | 50 | — | — | |
| IFalpha=0.1, merging_method=TIES-merging, DARE_augmented=true2024.10 | 50 | — | — | |
| MERGEPRINTalpha=0.3, merging_method=TIES-merging, DARE_augmented=false2024.10 | 50 | — | — | |
| TRAPalpha=0.5, merging_method=TIES-merging, DARE_augmented=false2024.10 | 49 | — | — | |
| IFalpha=0.1, merging_method=Task Arithmetic, DARE_augmented=false2024.10 | 49 | — | — | |
| IFalpha=0.1, merging_method=Task Arithmetic, DARE_augmented=true2024.10 | 49 | — | — | |
| IFalpha=0.3, merging_method=TIES-merging, DARE_augmented=true2024.10 | 49 | — | — | |
| MERGEPRINTalpha=0.1, merging_method=Task Arithmetic, DARE_augmented=false2024.10 | 49 | — | — | |
| MERGEPRINTalpha=0.1, merging_method=Task Arithmetic, DARE_augmented=true2024.10 | 49 | — | — | |
| MERGEPRINTalpha=0.7, merging_method=TIES-merging, DARE_augmented=true2024.10 | 49 | — | — | |
| IFalpha=0.1, merging_method=TIES-merging, DARE_augmented=false2024.10 | 48 | — | — | |
| IFalpha=0.3, merging_method=TIES-merging, DARE_augmented=false2024.10 | 48 | — | — | |
| IFalpha=0.5, merging_method=TIES-merging, DARE_augmented=true2024.10 | 48 | — | — | |
| MERGEPRINTalpha=0.5, merging_method=TIES-merging, DARE_augmented=false2024.10 | 48 | — | — | |
| TRAPalpha=0.9, merging_method=TIES-merging, DARE_augmented=true2024.10 | 47 | — | — | |
| MERGEPRINTalpha=0.9, merging_method=TIES-merging, DARE_augmented=true2024.10 | 47 | — | — | |
| TRAPalpha=0.3, merging_method=Task Arithmetic, DARE_augmented=false2024.10 | 46 | — | — | |
| TRAPalpha=0.3, merging_method=Task Arithmetic, DARE_augmented=true2024.10 | 46 | — | — | |
| IFalpha=0.5, merging_method=TIES-merging, DARE_augmented=false2024.10 | 46 | — | — | |
| TRAPalpha=0.7, merging_method=TIES-merging, DARE_augmented=false2024.10 | 45 | — | — | |
| IFalpha=0.3, merging_method=Task Arithmetic, DARE_augmented=false2024.10 | 45 | — | — | |
| IFalpha=0.3, merging_method=Task Arithmetic, DARE_augmented=true2024.10 | 45 | — | — | |
| IFalpha=0.7, merging_method=TIES-merging, DARE_augmented=true2024.10 | 45 | — | — | |
| MERGEPRINTalpha=0.3, merging_method=Task Arithmetic, DARE_augmented=false2024.10 | 45 | — | — | |
| MERGEPRINTalpha=0.3, merging_method=Task Arithmetic, DARE_augmented=true2024.10 | 45 | — | — | |
| MERGEPRINTalpha=0.7, merging_method=TIES-merging, DARE_augmented=false2024.10 | 45 | — | — | |
| TRAPalpha=0.5, merging_method=Task Arithmetic, DARE_augmented=false2024.10 | 44 | — | — | |
| TRAPalpha=0.5, merging_method=Task Arithmetic, DARE_augmented=true2024.10 | 44 | — | — | |
| MERGEPRINTalpha=0.5, merging_method=Task Arithmetic, DARE_augmented=false2024.10 | 44 | — | — | |
| MERGEPRINTalpha=0.5, merging_method=Task Arithmetic, DARE_augmented=true2024.10 | 44 | — | — | |
| MERGEPRINTalpha=0.9, merging_method=TIES-merging, DARE_augmented=false2024.10 | 44 | — | — | |
| TRAPalpha=0.7, merging_method=Task Arithmetic, DARE_augmented=false2024.10 | 43 | — | — | |
| TRAPalpha=0.7, merging_method=Task Arithmetic, DARE_augmented=true2024.10 | 43 | — | — | |
| TRAPalpha=0.9, merging_method=TIES-merging, DARE_augmented=false2024.10 | 43 | — | — | |
| IFalpha=0.5, merging_method=Task Arithmetic, DARE_augmented=false2024.10 | 43 | — | — | |
| IFalpha=0.5, merging_method=Task Arithmetic, DARE_augmented=true2024.10 | 43 | — | — | |
| IFalpha=0.7, merging_method=TIES-merging, DARE_augmented=false2024.10 | 43 | — | — | |
| IFalpha=0.9, merging_method=TIES-merging, DARE_augmented=false2024.10 | 43 | — | — | |
| IFalpha=0.9, merging_method=TIES-merging, DARE_augmented=true2024.10 | 43 | — | — | |
| MERGEPRINTalpha=0.7, merging_method=Task Arithmetic, DARE_augmented=false2024.10 | 43 | — | — | |
| MERGEPRINTalpha=0.9, merging_method=Task Arithmetic, DARE_augmented=false2024.10 | 43 | — | — | |
| MERGEPRINTalpha=0.7, merging_method=Task Arithmetic, DARE_augmented=true2024.10 | 43 | — | — | |
| MERGEPRINTalpha=0.9, merging_method=Task Arithmetic, DARE_augmented=true2024.10 | 43 | — | — | |
| TRAPalpha=0.9, merging_method=Task Arithmetic, DARE_augmented=false2024.10 | 42 | — | — | |
| TRAPalpha=0.9, merging_method=Task Arithmetic, DARE_augmented=true2024.10 | 42 | — | — | |
| IFalpha=0.7, merging_method=Task Arithmetic, DARE_augmented=false2024.10 | 42 | — | — | |
| IFalpha=0.9, merging_method=Task Arithmetic, DARE_augmented=false2024.10 | 42 | — | — | |
| IFalpha=0.7, merging_method=Task Arithmetic, DARE_augmented=true2024.10 | 42 | — | — | |
| IFalpha=0.9, merging_method=Task Arithmetic, DARE_augmented=true2024.10 | 42 | — | — | |
| Model Surgery2025.12 | 39.9 | — | — | |
| DeVec2025.12 | 32.1 | — | — | |
| Claude-3.7-SonnetAttack=None2026.06 | — | 0 | 19.49 | |
| DeepSeek-V3Attack=None2026.06 | — | 5.88 | 39.6 | |
| GHOSTWRITERTarget Model=GPT-4o, Attacker LM=GPT-4o-mini2026.06 | — | 17.65 | 57.43 | |
| GHOSTWRITERTarget Model=Claude-3.7-Sonnet, Attacker LM=GPT-4o-mini2026.06 | — | 23.53 | 67.33 | |
| GHOSTWRITERTarget Model=DeepSeek-V3, Attacker LM=GPT-4o-mini2026.06 | — | 38.24 | 70.3 | |
| GHOSTWRITERTarget Model=GPT-4o, Attacker LM=Qwen-2.5-7B2026.06 | — | 72.5 | 62.71 | |
| GHOSTWRITERTarget Model=Claude-3.7-Sonnet, Attacker LM=Qwen-2.5-7B2026.06 | — | 57.97 | 63.56 | |
| GHOSTWRITERTarget Model=DeepSeek-V3, Attacker LM=Qwen-2.5-7B2026.06 | — | 100 | 69.49 | |
| GPT-4oAttack=None2026.06 | — | 0 | 30.51 |