List your company
ResearchFormerly WizWand

Safety Evaluation on XSTest Unsafe

0False Refusal Rate (FR)No Defense
-3.821.8547.573.15Aug 6, 2025Sep 28, 2025Nov 21, 2025Jan 14, 2026Mar 9, 2026May 2, 2026Jun 25, 2026
Updated 4mo ago

Evaluation results

MethodLinks
No DefenseModel=R1-Qwen-7B, Output Component=Reasoning2025.0803268
SafeDecodingModel=R1-Qwen-7B, Output Component=Reasoning2025.0802872
ParaphraseModel=R1-Qwen-7B, Output Component=Reasoning2025.0802476
Self-ReminderModel=R1-Qwen-7B, Output Component=Reasoning2025.080892
SmoothLLMModel=R1-Qwen-7B, Output Component=Reasoning2025.0802080
No DefenseModel=R1-Llama-8B, Output Component=Reasoning2025.0802674
SafeDecodingModel=R1-Llama-8B, Output Component=Reasoning2025.0802476
No DefenseModel=R1-Qwen-14B, Evaluation Component=Reasoning2025.0801684
ParaphraseModel=R1-Qwen-14B, Evaluation Component=Reasoning2025.0803268
ParaphraseModel=R1-Qwen-32B, Evaluation Component=Reasoning2025.0804060
Self-ReminderModel=R1-Qwen-32B, Evaluation Component=Reasoning2025.0801486
Random featureModel=Qwen3-1.7B-it2026.060——
No DefenseModel=R1-Llama-8B, Output Component=Final Answer2025.0825444
SafeDecodingModel=R1-Llama-8B, Output Component=Final Answer2025.0825048
ParaphraseModel=R1-Llama-8B, Output Component=Reasoning2025.0823860
ParaphraseModel=R1-Llama-8B, Output Component=Final Answer2025.0825444
Self-ReminderModel=R1-Llama-8B, Output Component=Reasoning2025.0821286
No DefenseModel=R1-Qwen-32B, Evaluation Component=Reasoning2025.0822474
ReasoningGuardModel=R1-Qwen-32B, Evaluation Component=Reasoning2025.082296
Random featureModel=Qwen3-4B-it2026.062.3——
ParaphraseModel=R1-Qwen-7B, Output Component=Final Answer2025.0845244
SmoothLLMModel=R1-Qwen-7B, Output Component=Final Answer2025.0843660
SmoothLLMModel=R1-Qwen-14B, Evaluation Component=Reasoning2025.0842274
ReasoningGuardModel=R1-Qwen-14B, Evaluation Component=Reasoning2025.084096
SmoothLLMModel=R1-Qwen-32B, Evaluation Component=Reasoning2025.0843066
SafeDecodingModel=R1-Qwen-7B, Output Component=Final Answer2025.0864846
SmoothLLMModel=R1-Llama-8B, Output Component=Reasoning2025.0863262
ReasoningGuardModel=R1-Llama-8B, Output Component=Reasoning2025.086292
No DefenseModel=R1-Qwen-14B, Evaluation Component=Final Answer2025.0864648
SAFEPATH-ZSModel=R1-Qwen-14B, Evaluation Component=Reasoning2025.0861480
SmoothLLMModel=R1-Qwen-32B, Evaluation Component=Final Answer2025.0864450
SAFEPATH-ZSModel=R1-Qwen-32B, Evaluation Component=Reasoning2025.0861282
ReasoningGuardModel=R1-Qwen-7B, Output Component=Reasoning2025.088092
ParaphraseModel=R1-Qwen-32B, Evaluation Component=Final Answer2025.0885240
Self-ReminderModel=R1-Qwen-14B, Evaluation Component=Reasoning2025.08101080
SmoothLLMModel=R1-Qwen-14B, Evaluation Component=Final Answer2025.08104446
No DefenseModel=R1-Qwen-7B, Output Component=Final Answer2025.08124246
SAFEPATH-ZSModel=R1-Llama-8B, Output Component=Reasoning2025.08121672
No DefenseModel=R1-Qwen-32B, Evaluation Component=Final Answer2025.08124642
SmoothLLMModel=R1-Llama-8B, Output Component=Final Answer2025.08144046
SAFEPATH-ZSModel=R1-Qwen-7B, Output Component=Reasoning2025.0816678
ParaphraseModel=R1-Qwen-14B, Evaluation Component=Final Answer2025.08184834
SAFEPATH-ZSModel=R1-Qwen-32B, Evaluation Component=Final Answer2025.08223048
SafeKeyModel=R1-Llama-8B, Output Component=Reasoning2025.0824076
SAFEPATH-FTModel=R1-Qwen-7B, Output Component=Reasoning2025.08261262
Self-ReminderModel=R1-Llama-8B, Output Component=Final Answer2025.08263440
SAFEPATH-ZSModel=R1-Qwen-14B, Evaluation Component=Final Answer2025.08262846
SAFEPATH-ZSModel=R1-Llama-8B, Output Component=Final Answer2025.08283240
Self-ReminderModel=R1-Qwen-14B, Evaluation Component=Final Answer2025.08282448
ReasoningGuardModel=R1-Qwen-14B, Evaluation Component=Final Answer2025.08282250
Self-ReminderModel=R1-Qwen-32B, Evaluation Component=Final Answer2025.08282448
SAFEPATH-FTModel=R1-Qwen-7B, Output Component=Final Answer2025.08301654
ThinkingIModel=R1-Qwen-32B, Evaluation Component=Reasoning2025.08361054
ReasoningGuardModel=R1-Qwen-32B, Evaluation Component=Final Answer2025.08362044
RealSafe-R1Model=R1-Llama-8B, Output Component=Reasoning2025.0838260
ReasoningGuardModel=R1-Llama-8B, Output Component=Final Answer2025.08382042
SafeKeyModel=R1-Qwen-7B, Output Component=Reasoning2025.0840258
Self-ReminderModel=R1-Qwen-7B, Output Component=Final Answer2025.08401446
SAFEPATH-ZSModel=R1-Qwen-7B, Output Component=Final Answer2025.08402238
SafeKeyModel=R1-Qwen-14B, Evaluation Component=Reasoning2025.0840060
ThinkingIModel=R1-Qwen-32B, Evaluation Component=Final Answer2025.08403030
RealSafe-R1Model=R1-Qwen-7B, Output Component=Reasoning2025.0842058
SAFEPATH-FTModel=R1-Llama-8B, Output Component=Reasoning2025.0842652
SAFEPATH-FTModel=R1-Llama-8B, Output Component=Final Answer2025.08422236
RealSafe-R1Model=R1-Qwen-32B, Evaluation Component=Reasoning2025.0844254
RealSafe-R1Model=R1-Qwen-14B, Evaluation Component=Reasoning2025.0846252
SafeKeyModel=R1-Llama-8B, Output Component=Final Answer2025.0848250
ReasoningGuardModel=R1-Qwen-7B, Output Component=Final Answer2025.08502426
SafeKeyModel=R1-Qwen-7B, Output Component=Final Answer2025.0856836
ThinkingIModel=R1-Qwen-14B, Evaluation Component=Reasoning2025.08561034
ThinkingIModel=R1-Qwen-14B, Evaluation Component=Final Answer2025.08562024
No SteeringModel=Qwen3-1.7B-it2026.0659.5——
SafeKeyModel=R1-Qwen-14B, Evaluation Component=Final Answer2025.0860832
ThinkingIModel=R1-Llama-8B, Output Component=Final Answer2025.08641224
No SteeringModel=Qwen3-4B-it2026.0668.5——
ThinkingIModel=R1-Llama-8B, Output Component=Reasoning2025.0874422
ThinkingIModel=R1-Qwen-7B, Output Component=Final Answer2025.08761014
Target featureModel=Qwen3-1.7B-it2026.0676——
ThinkingIModel=R1-Qwen-7B, Output Component=Reasoning2025.0878220
RealSafe-R1Model=R1-Llama-8B, Output Component=Final Answer2025.0882018
RealSafe-R1Model=R1-Qwen-32B, Evaluation Component=Final Answer2025.0882018
RealSafe-R1Model=R1-Qwen-14B, Evaluation Component=Final Answer2025.0886410
RealSafe-R1Model=R1-Qwen-7B, Output Component=Final Answer2025.0890010
Target featureModel=Qwen3-4B-it2026.0695——