Safety Evaluation on AdvBench (Harmfulness Rate by Step)
0Reasoning Harmfulness RateRealSafe-R1
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| RealSafe-R1Model=R1-Qwen-7B, Method Type=Training-based2025.08 | 0 | 0 | — | |
| SAFEPATH-FTModel=R1-Qwen-7B, Method Type=Training-based2025.08 | 0 | 2 | — | |
| SafeKeyModel=R1-Qwen-7B, Method Type=Training-based2025.08 | 0 | 0 | — | |
| ThinkingIModel=R1-Qwen-7B, Method Type=Training-free2025.08 | 0 | 2 | — | |
| RealSafe-R1Model=R1-Llama-8B, Method Type=Training-based2025.08 | 0 | 0 | — | |
| SAFEPATH-FTModel=R1-Llama-8B, Method Type=Training-based2025.08 | 0 | 40 | — | |
| SafeKeyModel=R1-Llama-8B, Method Type=Training-based2025.08 | 0 | 0 | — | |
| ThinkingIModel=R1-Llama-8B, Method Type=Training-free2025.08 | 0 | 0 | — | |
| ReasoningGuardModel=R1-Llama-8B, Method Type=Training-free2025.08 | 0 | 0 | — | |
| ReasoningGuardModel Family=Phi-4-reasoning2025.08 | 1.02 | 1 | — | |
| ParaphraseModel Family=Phi-4-reasoning2025.08 | 1.08 | 1.28 | — | |
| Self-ReminderModel Family=Phi-4-reasoning2025.08 | 1.1 | 1 | — | |
| ThinkingIModel Family=Phi-4-reasoning2025.08 | 1.12 | 1.02 | — | |
| No DefenseModel Family=Phi-4-reasoning2025.08 | 1.14 | 1 | — | |
| ReasoningGuardModel Family=Qwen3-4B-Thinking2025.08 | 1.14 | 1 | — | |
| No DefenseModel Family=Qwen3-4B-Thinking2025.08 | 1.18 | 1.08 | — | |
| ThinkingIModel Family=Qwen3-4B-Thinking2025.08 | 1.2 | 1.04 | — | |
| SAFEPATH-ZSModel Family=Phi-4-reasoning2025.08 | 1.22 | 1 | — | |
| ReasoningGuardModel Family=Average2025.08 | 1.22 | 1 | — | |
| Self-ReminderModel Family=Qwen3-4B-Thinking2025.08 | 1.24 | 1 | — | |
| ThinkingIModel Family=Average2025.08 | 1.24 | 1.03 | — | |
| ParaphraseModel Family=Qwen3-4B-Thinking2025.08 | 1.3 | 1.12 | — | |
| SmoothLLMModel Family=Phi-4-reasoning2025.08 | 1.32 | 1.02 | — | |
| Self-ReminderModel Family=Average2025.08 | 1.33 | 1 | — | |
| SAFEPATH-ZSModel Family=Qwen3-4B-Thinking2025.08 | 1.36 | 1.12 | — | |
| SAFEPATH-ZSModel Family=Average2025.08 | 1.39 | 1.07 | — | |
| ThinkingIModel Family=QwQ-32B2025.08 | 1.4 | 1.02 | — | |
| ParaphraseModel Family=Average2025.08 | 1.47 | 1.29 | — | |
| SmoothLLMModel Family=Qwen3-4B-Thinking2025.08 | 1.48 | 1.14 | — | |
| ReasoningGuardModel Family=QwQ-32B2025.08 | 1.5 | 1 | — | |
| No DefenseModel Family=Average2025.08 | 1.53 | 1.13 | — | |
| SAFEPATH-ZSModel Family=QwQ-32B2025.08 | 1.58 | 1.08 | — | |
| Self-ReminderModel Family=QwQ-32B2025.08 | 1.66 | 1 | — | |
| SmoothLLMModel Family=Average2025.08 | 1.71 | 1.22 | — | |
| ReasoningGuardModel=R1-Qwen-7B, Method Type=Training-free2025.08 | 2 | 2 | — | |
| SAFEPATH-ZSModel=R1-Llama-8B, Method Type=Training-free2025.08 | 2 | 4 | — | |
| ParaphraseModel Family=QwQ-32B2025.08 | 2.02 | 1.46 | — | |
| No DefenseModel Family=QwQ-32B2025.08 | 2.28 | 1.3 | — | |
| SmoothLLMModel Family=QwQ-32B2025.08 | 2.32 | 1.5 | — | |
| Self-ReminderModel=R1-Qwen-7B, Method Type=Training-free2025.08 | 6 | 6 | — | |
| Self-ReminderModel=R1-Llama-8B, Method Type=Training-free2025.08 | 6 | 2 | — | |
| SAFEPATH-ZSModel=R1-Qwen-7B, Method Type=Training-free2025.08 | 12 | 6 | — | |
| ParaphraseModel=R1-Llama-8B, Method Type=Training-free2025.08 | 14 | 14 | — | |
| No DefenseModel=R1-Llama-8B, Method Type=Baseline2025.08 | 26 | 22 | — | |
| SmoothLLMModel=R1-Llama-8B, Method Type=Training-free2025.08 | 28 | 24 | — | |
| SmoothLLMModel=R1-Qwen-7B, Method Type=Training-free2025.08 | 30 | 20 | — | |
| SafeDecodingModel=R1-Llama-8B, Method Type=Training-based2025.08 | 30 | 28 | — | |
| SafeDecodingModel=R1-Qwen-7B, Method Type=Training-based2025.08 | 38 | 40 | — | |
| No DefenseModel=R1-Qwen-7B, Method Type=Baseline2025.08 | 40 | 32 | — | |
| ParaphraseModel=R1-Qwen-7B, Method Type=Training-free2025.08 | 40 | 40 | — | |
| No DefenseModel=R1-Qwen-14B2025.08 | — | — | 42 | |
| No DefenseModel=R1-Qwen-32B2025.08 | — | — | 22 | |
| ParaphraseModel=R1-Qwen-14B2025.08 | — | — | 20 | |
| ParaphraseModel=R1-Qwen-32B2025.08 | — | — | 14 | |
| RealSafe-R1Model=R1-Qwen-14B2025.08 | — | — | 0 | |
| RealSafe-R1Model=R1-Qwen-32B2025.08 | — | — | 0 | |
| ReasoningGuardModel=R1-Qwen-14B2025.08 | — | — | 0 | |
| ReasoningGuardModel=R1-Qwen-32B2025.08 | — | — | 0 | |
| SafeKeyModel=R1-Qwen-14B2025.08 | — | — | 0 | |
| SAFEPATH-ZSModel=R1-Qwen-14B2025.08 | — | — | 4 | |
| SAFEPATH-ZSModel=R1-Qwen-32B2025.08 | — | — | 2 | |
| Self-ReminderModel=R1-Qwen-14B2025.08 | — | — | 2 | |
| Self-ReminderModel=R1-Qwen-32B2025.08 | — | — | 0 | |
| SmoothLLMModel=R1-Qwen-14B2025.08 | — | — | 18 | |
| SmoothLLMModel=R1-Qwen-32B2025.08 | — | — | 10 | |
| ThinkingIModel=R1-Qwen-14B2025.08 | — | — | 0 | |
| ThinkingIModel=R1-Qwen-32B2025.08 | — | — | 0 |