List your company
ResearchFormerly WizWand

Safety Refusal Evaluation on PolyRefuse 1.0 (sw)

96.7Harmful Refusal RateOurs (HRL + 32 LRL)
17.97238.41158.8579.289May 31, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Ours (HRL + 32 LRL)Backbone=Gemma-2-9B, Steering Method=HRL + 32 LRL2026.0596.73.693.1
Ours (HRL)Backbone=Gemma-2-9B, Steering Method=HRL2026.0595.53.891.7
CAST (HRL + 32 LRL)Backbone=Gemma-2-9B, Steering Method=CAST (HRL + 32 LRL)2026.0588.3187.3
CAST (Original)Backbone=Gemma-2-9B, Steering Method=CAST (Original)2026.0587.6186.6
AdaSteer (Original)Backbone=Gemma-2-9B, Steering Method=AdaSteer (Original)2026.0586.4482.4
Gemma-2-9BBackbone=Gemma-2-9B, Steering Method=None2026.0586.2185.2
AdaSteer (HRL + 32 LRL)Backbone=Llama-3.1-8B-Instruct, Steering Method=AdaSteer (HRL + 32 LRL)2026.0583.479.24.2
AdaSteer (HRL + 32 LRL)Backbone=Gemma-2-9B, Steering Method=AdaSteer (HRL + 32 LRL)2026.0573.39.463.9
Ours (HRL)Backbone=Qwen2.5-7B-Instruct, Steering Method=HRL2026.0556.610.646
Ours (HRL + 32 LRL)Backbone=Qwen2.5-7B-Instruct, Steering Method=HRL + 32 LRL2026.0556.610.446.2
Ours (HRL + 32 LRL)Backbone=Llama-3.1-8B-Instruct, Steering Method=HRL + 32 LRL2026.0553.71.452.3
CAST (Original)Backbone=Llama-3.1-8B-Instruct, Steering Method=CAST (Original)2026.055037.612.4
AdaSteer (Original)Backbone=Llama-3.1-8B-Instruct, Steering Method=AdaSteer (Original)2026.0546.2541.2
CAST (HRL + 32 LRL)Backbone=Llama-3.1-8B-Instruct, Steering Method=CAST (HRL + 32 LRL)2026.0545.110.434.7
AdaSteer (Original)Backbone=Qwen2.5-7B-Instruct, Steering Method=AdaSteer (Original)2026.0543.525.817.7
Llama-3.1-8B-InstructBackbone=Llama-3.1-8B-Instruct, Steering Method=None2026.0538.60.837.8
CAST (Original)Backbone=Qwen2.5-7B-Instruct, Steering Method=CAST (Original)2026.0529.212.816.4
Ours (HRL)Backbone=Llama-3.1-8B-Instruct, Steering Method=HRL2026.0529128
AdaSteer (HRL + 32 LRL)Backbone=Qwen2.5-7B-Instruct, Steering Method=AdaSteer (HRL + 32 LRL)2026.0526.716.410.3
Qwen2.5-7B-InstructBackbone=Qwen2.5-7B-Instruct, Steering Method=None2026.0522.9913.9
CAST (HRL + 32 LRL)Backbone=Qwen2.5-7B-Instruct, Steering Method=CAST (HRL + 32 LRL)2026.0521714