List your company
ResearchFormerly WizWand

Safety Refusal Evaluation on PolyRefuse 1.0 (km)

90.9Harmful Refusal RateOurs (HRL + 32 LRL)
22.67640.38858.175.812May 31, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Ours (HRL + 32 LRL)Backbone=Gemma-2-9B, Steering Method=HRL + 32 LRL2026.0590.98.282.7
AdaSteer (HRL + 32 LRL)Backbone=Llama-3.1-8B-Instruct, Steering Method=AdaSteer (HRL + 32 LRL)2026.0587.287.8-0.6
Ours (HRL)Backbone=Gemma-2-9B, Steering Method=HRL2026.0586.26.479.8
CAST (HRL + 32 LRL)Backbone=Gemma-2-9B, Steering Method=CAST (HRL + 32 LRL)2026.0581.1675.1
AdaSteer (Original)Backbone=Qwen2.5-7B-Instruct, Steering Method=AdaSteer (Original)2026.0579.95326.9
CAST (Original)Backbone=Gemma-2-9B, Steering Method=CAST (Original)2026.0579.2673.2
Ours (HRL + 32 LRL)Backbone=Qwen2.5-7B-Instruct, Steering Method=HRL + 32 LRL2026.0574.315.658.7
Gemma-2-9BBackbone=Gemma-2-9B, Steering Method=None2026.0574.35.468.9
AdaSteer (Original)Backbone=Gemma-2-9B, Steering Method=AdaSteer (Original)2026.05741757
CAST (Original)Backbone=Llama-3.1-8B-Instruct, Steering Method=CAST (Original)2026.0570.65812.6
AdaSteer (HRL + 32 LRL)Backbone=Gemma-2-9B, Steering Method=AdaSteer (HRL + 32 LRL)2026.0563.349.613.7
Ours (HRL + 32 LRL)Backbone=Llama-3.1-8B-Instruct, Steering Method=HRL + 32 LRL2026.0558.24.853.4
AdaSteer (Original)Backbone=Llama-3.1-8B-Instruct, Steering Method=AdaSteer (Original)2026.0555.813.642.2
Ours (HRL)Backbone=Qwen2.5-7B-Instruct, Steering Method=HRL2026.0555.68.647
CAST (HRL + 32 LRL)Backbone=Llama-3.1-8B-Instruct, Steering Method=CAST (HRL + 32 LRL)2026.0553.123.629.5
Llama-3.1-8B-InstructBackbone=Llama-3.1-8B-Instruct, Steering Method=None2026.0544.85.239.6
CAST (Original)Backbone=Qwen2.5-7B-Instruct, Steering Method=CAST (Original)2026.0543.41330.4
AdaSteer (HRL + 32 LRL)Backbone=Qwen2.5-7B-Instruct, Steering Method=AdaSteer (HRL + 32 LRL)2026.0540.713.227.5
Qwen2.5-7B-InstructBackbone=Qwen2.5-7B-Instruct, Steering Method=None2026.0531.1724.1
CAST (HRL + 32 LRL)Backbone=Qwen2.5-7B-Instruct, Steering Method=CAST (HRL + 32 LRL)2026.0529.25.823.4
Ours (HRL)Backbone=Llama-3.1-8B-Instruct, Steering Method=HRL2026.0525.33.621.7