Safety Refusal Evaluation on PolyRefuse 1.0 (my)
91.3Harmful Refusal RateAdaSteer (HRL + 32 LRL)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| AdaSteer (HRL + 32 LRL)Backbone=Llama-3.1-8B-Instruct, Steering Method=AdaSteer (HRL + 32 LRL)2026.05 | 91.3 | 92.2 | -0.9 | |
| Ours (HRL + 32 LRL)Backbone=Gemma-2-9B, Steering Method=HRL + 32 LRL2026.05 | 85.1 | 5.8 | 79.3 | |
| AdaSteer (Original)Backbone=Qwen2.5-7B-Instruct, Steering Method=AdaSteer (Original)2026.05 | 82 | 66.2 | 15.8 | |
| Ours (HRL)Backbone=Gemma-2-9B, Steering Method=HRL2026.05 | 81.6 | 4.4 | 77.2 | |
| CAST (Original)Backbone=Llama-3.1-8B-Instruct, Steering Method=CAST (Original)2026.05 | 77.6 | 57.4 | 20.2 | |
| CAST (Original)Backbone=Gemma-2-9B, Steering Method=CAST (Original)2026.05 | 55.6 | 4.4 | 51.2 | |
| CAST (HRL + 32 LRL)Backbone=Gemma-2-9B, Steering Method=CAST (HRL + 32 LRL)2026.05 | 55.4 | 4.2 | 51.2 | |
| Gemma-2-9BBackbone=Gemma-2-9B, Steering Method=None2026.05 | 54.9 | 3.8 | 51.1 | |
| CAST (HRL + 32 LRL)Backbone=Llama-3.1-8B-Instruct, Steering Method=CAST (HRL + 32 LRL)2026.05 | 54.5 | 26.4 | 28.1 | |
| AdaSteer (Original)Backbone=Llama-3.1-8B-Instruct, Steering Method=AdaSteer (Original)2026.05 | 51.6 | 17.4 | 34.2 | |
| Ours (HRL + 32 LRL)Backbone=Qwen2.5-7B-Instruct, Steering Method=HRL + 32 LRL2026.05 | 51 | 10.8 | 40.2 | |
| AdaSteer (Original)Backbone=Gemma-2-9B, Steering Method=AdaSteer (Original)2026.05 | 50.7 | 14.6 | 36.1 | |
| AdaSteer (HRL + 32 LRL)Backbone=Gemma-2-9B, Steering Method=AdaSteer (HRL + 32 LRL)2026.05 | 48.1 | 38.4 | 9.7 | |
| Ours (HRL + 32 LRL)Backbone=Llama-3.1-8B-Instruct, Steering Method=HRL + 32 LRL2026.05 | 46.7 | 7.2 | 39.5 | |
| Ours (HRL)Backbone=Qwen2.5-7B-Instruct, Steering Method=HRL2026.05 | 46 | 7.8 | 38.2 | |
| Llama-3.1-8B-InstructBackbone=Llama-3.1-8B-Instruct, Steering Method=None2026.05 | 40.2 | 8 | 32.2 | |
| AdaSteer (HRL + 32 LRL)Backbone=Qwen2.5-7B-Instruct, Steering Method=AdaSteer (HRL + 32 LRL)2026.05 | 38.1 | 18.6 | 19.5 | |
| CAST (Original)Backbone=Qwen2.5-7B-Instruct, Steering Method=CAST (Original)2026.05 | 37.1 | 10.4 | 26.7 | |
| Ours (HRL)Backbone=Llama-3.1-8B-Instruct, Steering Method=HRL2026.05 | 33.4 | 7.6 | 25.8 | |
| CAST (HRL + 32 LRL)Backbone=Qwen2.5-7B-Instruct, Steering Method=CAST (HRL + 32 LRL)2026.05 | 30.9 | 8.4 | 22.5 | |
| Qwen2.5-7B-InstructBackbone=Qwen2.5-7B-Instruct, Steering Method=None2026.05 | 29.5 | 8.4 | 21.1 |