List your company
ResearchFormerly WizWand

Safety Refusal Evaluation on PolyRefuse yo 1.0

17.7Harmful Refusal RateOurs (HRL)
14.73634.74354.7574.757May 31, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Ours (HRL)Backbone=Llama-3.1-8B-Instruct, Steering Method=HRL2026.0517.78.69.1
Llama-3.1-8B-InstructBackbone=Llama-3.1-8B-Instruct, Steering Method=None2026.0520.89.811
AdaSteer (Original)Backbone=Llama-3.1-8B-Instruct, Steering Method=AdaSteer (Original)2026.0528.315.412.9
Ours (HRL + 32 LRL)Backbone=Llama-3.1-8B-Instruct, Steering Method=HRL + 32 LRL2026.0532.79.223.5
Qwen2.5-7B-InstructBackbone=Qwen2.5-7B-Instruct, Steering Method=None2026.0543.539.83.7
AdaSteer (HRL + 32 LRL)Backbone=Qwen2.5-7B-Instruct, Steering Method=AdaSteer (HRL + 32 LRL)2026.0544.847.8-3
CAST (Original)Backbone=Qwen2.5-7B-Instruct, Steering Method=CAST (Original)2026.0546.936.210.7
CAST (HRL + 32 LRL)Backbone=Qwen2.5-7B-Instruct, Steering Method=CAST (HRL + 32 LRL)2026.055040.89.2
CAST (HRL + 32 LRL)Backbone=Llama-3.1-8B-Instruct, Steering Method=CAST (HRL + 32 LRL)2026.0554.743.810.9
AdaSteer (HRL + 32 LRL)Backbone=Gemma-2-9B, Steering Method=AdaSteer (HRL + 32 LRL)2026.0562.472-9.6
Gemma-2-9BBackbone=Gemma-2-9B, Steering Method=None2026.0566.630.436.2
AdaSteer (Original)Backbone=Gemma-2-9B, Steering Method=AdaSteer (Original)2026.0570.539.431.1
CAST (Original)Backbone=Gemma-2-9B, Steering Method=CAST (Original)2026.0573.333.639.7
CAST (HRL + 32 LRL)Backbone=Gemma-2-9B, Steering Method=CAST (HRL + 32 LRL)2026.0573.631.442.2
Ours (HRL)Backbone=Gemma-2-9B, Steering Method=HRL2026.0576.219.656.6
CAST (Original)Backbone=Llama-3.1-8B-Instruct, Steering Method=CAST (Original)2026.0576.472.44
AdaSteer (Original)Backbone=Qwen2.5-7B-Instruct, Steering Method=AdaSteer (Original)2026.0579.277.22
Ours (HRL + 32 LRL)Backbone=Qwen2.5-7B-Instruct, Steering Method=HRL + 32 LRL2026.0587.248.638.6
Ours (HRL)Backbone=Qwen2.5-7B-Instruct, Steering Method=HRL2026.0587.648.439.2
Ours (HRL + 32 LRL)Backbone=Gemma-2-9B, Steering Method=HRL + 32 LRL2026.0590.425.664.8
AdaSteer (HRL + 32 LRL)Backbone=Llama-3.1-8B-Instruct, Steering Method=AdaSteer (HRL + 32 LRL)2026.0591.889.62.2