Safety Alignment Breaking Prevention on StrongREJECT
0Harmful Score (%)CTRAP
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| CTRAPModel=Qwen2.5-3B-Instruct, Harmful-RL version=w/o-attack2025.08 | 0 | 0.41 | |
| TokenBuncherModel=Qwen2.5-3B-Instruct, Harmful-RL version=w/o-attack2025.08 | 0 | 0 | |
| TokenBuncherModel=Qwen2.5-3B-Instruct, Harmful-RL version=PPO2025.08 | 0 | 1.88 | |
| CTRAPModel=Qwen2.5-7B-Instruct, Harmful-RL version=w/o-attack2025.08 | 0 | 0.35 | |
| TokenBuncherModel=Qwen2.5-7B-Instruct, Harmful-RL version=w/o-attack2025.08 | 0 | 0 | |
| TokenBuncherModel=Qwen2.5-7B-Instruct, Harmful-RL version=RLOO2025.08 | 0 | 0.62 | |
| TokenBuncherModel=Mistral-8B-Instruct, Harmful-RL version=w/o-attack2025.08 | 0 | 0 | |
| TokenBuncherModel=Mistral-8B-Instruct, Harmful-RL version=RLOO2025.08 | 0 | 0.73 | |
| TokenBuncherModel=Qwen2.5-3B-Instruct, Harmful-RL version=REIN++2025.08 | 0.32 | 1.54 | |
| TokenBuncherModel=Qwen2.5-7B-Instruct, Harmful-RL version=GRPO2025.08 | 0.32 | 0.32 | |
| TokenBuncherModel=Qwen2.5-3B-Instruct, Harmful-RL version=RLOO2025.08 | 0.64 | 0.73 | |
| CTRAPModel=Mistral-8B-Instruct, Harmful-RL version=w/o-attack2025.08 | 0.64 | 0.58 | |
| TokenBuncherModel=Mistral-8B-Instruct, Harmful-RL version=PPO2025.08 | 0.64 | 0.97 | |
| TokenBuncherModel=Qwen2.5-3B-Instruct, Harmful-RL version=GRPO2025.08 | 0.96 | 1.31 | |
| TokenBuncherModel=Qwen2.5-7B-Instruct, Harmful-RL version=REIN++2025.08 | 0.96 | 0.51 | |
| TokenBuncherModel=Qwen2.5-7B-Instruct, Harmful-RL version=PPO2025.08 | 1.28 | 0.22 | |
| RepNoiseModel=Qwen2.5-3B-Instruct, Harmful-RL version=w/o-attack2025.08 | 1.6 | 0.61 | |
| TokenBuncherModel=Mistral-8B-Instruct, Harmful-RL version=GRPO2025.08 | 1.6 | 0.25 | |
| RepNoiseModel=Qwen2.5-7B-Instruct, Harmful-RL version=w/o-attack2025.08 | 1.92 | 0.33 | |
| TokenBuncherModel=Mistral-8B-Instruct, Harmful-RL version=REIN++2025.08 | 1.92 | 0.22 | |
| BaseModel=Qwen2.5-7B-Instruct, Harmful-RL version=w/o-attack2025.08 | 5.75 | 1.45 | |
| RepNoiseModel=Mistral-8B-Instruct, Harmful-RL version=w/o-attack2025.08 | 5.75 | 0.3 | |
| BaseModel=Qwen2.5-3B-Instruct, Harmful-RL version=w/o-attack2025.08 | 9.27 | 1.98 | |
| BaseModel=Mistral-8B-Instruct, Harmful-RL version=w/o-attack2025.08 | 20.13 | 3.34 | |
| CTRAPModel=Qwen2.5-7B-Instruct, Harmful-RL version=RLOO2025.08 | 55.59 | 6.27 | |
| CTRAPModel=Qwen2.5-3B-Instruct, Harmful-RL version=RLOO2025.08 | 60.06 | 6.48 | |
| CTRAPModel=Mistral-8B-Instruct, Harmful-RL version=RLOO2025.08 | 60.06 | 9.51 | |
| RepNoiseModel=Qwen2.5-7B-Instruct, Harmful-RL version=RLOO2025.08 | 66.45 | 9.07 | |
| RepNoiseModel=Qwen2.5-3B-Instruct, Harmful-RL version=RLOO2025.08 | 66.77 | 9.07 | |
| BaseModel=Qwen2.5-3B-Instruct, Harmful-RL version=RLOO2025.08 | 68.37 | 9.17 | |
| CTRAPModel=Qwen2.5-7B-Instruct, Harmful-RL version=GRPO2025.08 | 68.69 | 6.44 | |
| RepNoiseModel=Qwen2.5-3B-Instruct, Harmful-RL version=PPO2025.08 | 69.01 | 9.09 | |
| CTRAPModel=Qwen2.5-7B-Instruct, Harmful-RL version=REIN++2025.08 | 69.65 | 5.81 | |
| BaseModel=Qwen2.5-7B-Instruct, Harmful-RL version=RLOO2025.08 | 69.97 | 9.2 | |
| RepNoiseModel=Mistral-8B-Instruct, Harmful-RL version=REIN++2025.08 | 71.25 | 8.43 | |
| RepNoiseModel=Qwen2.5-7B-Instruct, Harmful-RL version=PPO2025.08 | 72.52 | 8.5 | |
| BaseModel=Qwen2.5-7B-Instruct, Harmful-RL version=PPO2025.08 | 72.84 | 9.33 | |
| CTRAPModel=Qwen2.5-3B-Instruct, Harmful-RL version=PPO2025.08 | 73.48 | 8.13 | |
| CTRAPModel=Qwen2.5-3B-Instruct, Harmful-RL version=GRPO2025.08 | 73.48 | 6.92 | |
| CTRAPModel=Mistral-8B-Instruct, Harmful-RL version=PPO2025.08 | 73.48 | 7.32 | |
| CTRAPModel=Mistral-8B-Instruct, Harmful-RL version=GRPO2025.08 | 73.48 | 9.15 | |
| RepNoiseModel=Mistral-8B-Instruct, Harmful-RL version=PPO2025.08 | 74.44 | 6.33 | |
| RepNoiseModel=Mistral-8B-Instruct, Harmful-RL version=RLOO2025.08 | 74.44 | 9.55 | |
| BaseModel=Qwen2.5-3B-Instruct, Harmful-RL version=PPO2025.08 | 75.08 | 9.12 | |
| RepNoiseModel=Qwen2.5-7B-Instruct, Harmful-RL version=GRPO2025.08 | 75.4 | 9.23 | |
| BaseModel=Qwen2.5-3B-Instruct, Harmful-RL version=GRPO2025.08 | 75.52 | 9.46 | |
| RepNoiseModel=Qwen2.5-3B-Instruct, Harmful-RL version=GRPO2025.08 | 76.68 | 9.01 | |
| CTRAPModel=Qwen2.5-7B-Instruct, Harmful-RL version=PPO2025.08 | 76.68 | 7.92 | |
| BaseModel=Qwen2.5-7B-Instruct, Harmful-RL version=GRPO2025.08 | 78.59 | 9.03 | |
| BaseModel=Qwen2.5-3B-Instruct, Harmful-RL version=REIN++2025.08 | 79.23 | 9.55 | |
| BaseModel=Qwen2.5-7B-Instruct, Harmful-RL version=REIN++2025.08 | 79.87 | 9.11 | |
| RepNoiseModel=Mistral-8B-Instruct, Harmful-RL version=GRPO2025.08 | 79.87 | 9.61 | |
| BaseModel=Mistral-8B-Instruct, Harmful-RL version=RLOO2025.08 | 80.83 | 9.69 | |
| CTRAPModel=Qwen2.5-3B-Instruct, Harmful-RL version=REIN++2025.08 | 81.79 | 6.46 | |
| CTRAPModel=Mistral-8B-Instruct, Harmful-RL version=REIN++2025.08 | 81.79 | 8.88 | |
| BaseModel=Mistral-8B-Instruct, Harmful-RL version=GRPO2025.08 | 83.71 | 8.66 | |
| BaseModel=Mistral-8B-Instruct, Harmful-RL version=PPO2025.08 | 84.03 | 7.63 | |
| RepNoiseModel=Qwen2.5-3B-Instruct, Harmful-RL version=REIN++2025.08 | 84.35 | 8.37 | |
| RepNoiseModel=Qwen2.5-7B-Instruct, Harmful-RL version=REIN++2025.08 | 84.98 | 9.22 | |
| BaseModel=Mistral-8B-Instruct, Harmful-RL version=REIN++2025.08 | 86.27 | 9.67 |