Safety Evaluation on SorryBench (FFR Metrics)
54.5Reasoning Success Rate (FFR)No Defense
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| No DefenseModel=R1-Llama-8B2025.08 | 54.5 | 75 | — | |
| No DefenseModel=R1-Qwen-7B2025.08 | 43.2 | 86.4 | — | |
| ParaphraseModel Family=QwQ-32B2025.08 | 38.6 | 59.1 | — | |
| No DefenseModel Family=QwQ-32B2025.08 | 34.1 | 52.3 | — | |
| SmoothLLMModel Family=QwQ-32B2025.08 | 34.1 | 59.1 | — | |
| SAFEPATH-ZSModel Family=QwQ-32B2025.08 | 25 | 45.5 | — | |
| SAFEPATH-ZSModel Family=Qwen3-4B-Thinking2025.08 | 25 | 45.5 | — | |
| SmoothLLMModel Family=Average2025.08 | 19.7 | 40.9 | — | |
| ParaphraseModel Family=Average2025.08 | 18.9 | 40.1 | — | |
| Self-ReminderModel Family=Phi-4-reasoning2025.08 | 18.2 | 27.3 | — | |
| SAFEPATH-ZSModel Family=Average2025.08 | 18.2 | 27.6 | — | |
| ThinkingIModel Family=QwQ-32B2025.08 | 15.9 | 38.6 | — | |
| No DefenseModel Family=Average2025.08 | 15.2 | 36.4 | — | |
| Self-ReminderModel Family=Average2025.08 | 15.1 | 24.2 | — | |
| ParaphraseModel Family=Qwen3-4B-Thinking2025.08 | 13.6 | 34.1 | — | |
| Self-ReminderModel Family=QwQ-32B2025.08 | 13.6 | 38.6 | — | |
| Self-ReminderModel Family=Qwen3-4B-Thinking2025.08 | 13.6 | 6.8 | — | |
| SmoothLLMModel Family=Phi-4-reasoning2025.08 | 13.6 | 29.5 | — | |
| ReasoningGuardModel Family=QwQ-32B2025.08 | 13.6 | 36.4 | — | |
| SmoothLLMModel Family=Qwen3-4B-Thinking2025.08 | 11.4 | 34.1 | — | |
| No DefenseModel Family=Qwen3-4B-Thinking2025.08 | 9.1 | 27.3 | — | |
| ThinkingIModel Family=Qwen3-4B-Thinking2025.08 | 9.1 | 18.2 | — | |
| ThinkingIModel Family=Average2025.08 | 9.1 | 28 | — | |
| ReasoningGuardModel Family=Qwen3-4B-Thinking2025.08 | 6.8 | 11.4 | — | |
| ReasoningGuardModel Family=Average2025.08 | 6.8 | 23.5 | — | |
| ParaphraseModel Family=Phi-4-reasoning2025.08 | 4.5 | 27.2 | — | |
| SAFEPATH-ZSModel Family=Phi-4-reasoning2025.08 | 4.5 | 31.8 | — | |
| ReasoningGuardModel=R1-Llama-8B2025.08 | 2.3 | 15.9 | — | |
| No DefenseModel Family=Phi-4-reasoning2025.08 | 2.3 | 29.5 | — | |
| ThinkingIModel Family=Phi-4-reasoning2025.08 | 2.3 | 27.3 | — | |
| ReasoningGuardModel=R1-Qwen-7B2025.08 | 0 | 13.6 | — | |
| ReasoningGuardModel Family=Phi-4-reasoning2025.08 | 0 | 22.7 | — | |
| No DefenseModel=R1-Qwen-14B2025.08 | — | — | 72.7 | |
| No DefenseModel=R1-Qwen-32B2025.08 | — | — | 77.3 | |
| ParaphraseModel=R1-Qwen-14B2025.08 | — | — | 77.3 | |
| ParaphraseModel=R1-Qwen-32B2025.08 | — | — | 79.5 | |
| RealSafe-R1Model=R1-Qwen-14B2025.08 | — | — | 13.6 | |
| RealSafe-R1Model=R1-Qwen-32B2025.08 | — | — | 9.1 | |
| ReasoningGuardModel=R1-Qwen-14B2025.08 | — | — | 22.7 | |
| ReasoningGuardModel=R1-Qwen-32B2025.08 | — | — | 20.5 | |
| SafeKeyModel=R1-Qwen-14B2025.08 | — | — | 20.5 | |
| SAFEPATH-ZSModel=R1-Qwen-14B2025.08 | — | — | 47.7 | |
| SAFEPATH-ZSModel=R1-Qwen-32B2025.08 | — | — | 40.9 | |
| Self-ReminderModel=R1-Qwen-14B2025.08 | — | — | 38.6 | |
| Self-ReminderModel=R1-Qwen-32B2025.08 | — | — | 40.9 | |
| SmoothLLMModel=R1-Qwen-14B2025.08 | — | — | 70.5 | |
| SmoothLLMModel=R1-Qwen-32B2025.08 | — | — | 79.5 | |
| ThinkingIModel=R1-Qwen-14B2025.08 | — | — | 34.1 | |
| ThinkingIModel=R1-Qwen-32B2025.08 | — | — | 47.7 |