Refusal Steering on SALAD Alpaca HarmBench
99.8Target SuccessResidual Paving (oracle)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Residual Paving (oracle)Model=Gemma-3-4B-IT, Setting=local oracle, s = 8, Source=diagnostic2026.05 | 99.8 | 100 | 100 | |
| Residual PavingModel=Gemma-3-4B-IT, Setting=local thresholded veto, s = 8, Source=local2026.05 | 96 | 95.5 | 87.3 | |
| DIMModel=Llama-3-8B-Instruct, Setting=local recovered simple-steering, scale 1, Source=local2026.05 | 93.3 | 26.1 | 3.6 | |
| Refusal Steering WRMDModel=Qwen3-80B, Setting=source-reported, Source=source2026.05 | 76.2 | 97.1 | 99 | |
| Refusal Steering WRMDModel=Qwen3-4B Inst., Setting=source-reported, Source=source2026.05 | 67.4 | 99.6 | 55 | |
| Refusal Steering WRMDModel=Qwen3-4B Think., Setting=source-reported, Source=source2026.05 | 65.3 | 100.7 | 78 | |
| Direction OrthogonalizationModel=Gemma-3-4B-IT, Setting=local pairwise strict, scale 1, Source=local2026.05 | 23.3 | 81.7 | 83.3 | |
| DIMModel=Gemma-3-4B-IT, Setting=local pairwise strict, scale 1, Source=local2026.05 | 10 | 93.3 | 93.3 | |
| WRMD protocolModel=Gemma-3-4B-IT, Setting=local WRMD eval, no steering, Source=local2026.05 | 8.5 | 97 | 90 | |
| RepItModel=Gemma-3-4B-IT, Setting=local pairwise strict, scale 1, Source=local2026.05 | 6.7 | 83.3 | 93.3 | |
| WRMD protocolModel=Qwen3-4B Inst., Setting=local WRMD eval, no steering, Source=local2026.05 | 5.9 | 98 | 89 | |
| GoR-RDOModel=Gemma-3-4B-IT, Setting=local pairwise strict, scale 1, Source=local2026.05 | 5 | 95 | 93.3 | |
| GoR-RDOModel=Llama-3-8B-Instruct, Setting=local strict, scale 1, Source=local2026.05 | 1.7 | 96.5 | 94.9 | |
| RepItModel=Phi-4-mini-Instruct, Setting=local strict, scale 1, Source=local2026.05 | 1.7 | 98.8 | 99.1 | |
| DIMModel=Gemma-2-2B-IT, Setting=local strict, scale 1, Source=local2026.05 | 0 | 99.8 | 99.6 | |
| GoR-RDOModel=Gemma-2-2B-IT, Setting=local strict, scale 1, Source=local2026.05 | 0 | 100 | 100 | |
| DIMModel=Qwen2.5-14B-Instruct, Setting=local early-layer diagnostic, scale 1, Source=local2026.05 | 0 | 99.9 | 99.8 | |
| GoR-RDOModel=Qwen2.5-14B-Instruct, Setting=local early-layer diagnostic, scale 1, Source=local2026.05 | 0 | 99.9 | 99.9 |