Over-refusal Evaluation on XSTest Safe
1.6Over-refusal RateAdaCD
Evaluation Results
| Method | Links | |
|---|---|---|
| AdaCDBase Model=Llama32026.04 | 1.6 | |
| BaseBackbone Model=Qwen3-32B2026.06 | 1.6 | |
| SelfCDBase Model=Llama32026.04 | 2 | |
| AdaCDBase Model=Qwen32026.04 | 2 | |
| ST-DBackbone Model=Qwen3-8B2026.06 | 2 | |
| Star1Backbone Model=Qwen3-8B2026.06 | 2.4 | |
| ST-DBackbone Model=DeepSeek-R1-Distill-Llama-8B2026.06 | 2.4 | |
| SelfCDBase Model=Qwen32026.04 | 2.8 | |
| ST-SBackbone Model=DeepSeek-R1-Distill-Llama-8B2026.06 | 2.8 | |
| SafePathBackbone Model=Qwen3-32B2026.06 | 2.8 | |
| SurgicalBase Model=Qwen32026.04 | 3.2 | |
| BaseBackbone Model=Qwen3-8B2026.06 | 3.2 | |
| SafePathBackbone Model=Qwen3-8B2026.06 | 3.2 | |
| Star1Backbone Model=Qwen3-32B2026.06 | 3.2 | |
| ST-SBackbone Model=Qwen3-32B2026.06 | 3.2 | |
| ST-DBackbone Model=Qwen3-32B2026.06 | 3.2 | |
| Categorical SteeringBackbone=REFUSE-LLAMA, Method=Categorical Steering2026.03 | 3.6 | |
| SurgicalBase Model=Llama32026.04 | 3.6 | |
| DefaultBase Model=Qwen32026.04 | 4 | |
| BaseBackbone Model=DeepSeek-R1-Distill-Llama-8B2026.06 | 4 | |
| SSDBase Model=Qwen32026.04 | 4.8 | |
| ST-SBackbone Model=Qwen3-8B2026.06 | 4.8 | |
| Low-Rank CombinationBackbone=REFUSE-LLAMA, Method=Low-Rank Combination2026.03 | 5.2 | |
| DefaultBase Model=Llama32026.04 | 5.2 | |
| BaseBackbone Model=DeepSeek-R1-Distill-Llama-70B2026.06 | 5.2 | |
| PromptBase Model=Qwen32026.04 | 6 | |
| ST-SBackbone Model=DeepSeek-R1-Distill-Llama-70B2026.06 | 6 | |
| ST-DBackbone Model=DeepSeek-R1-Distill-Llama-70B2026.06 | 6.8 | |
| Star1Backbone Model=DeepSeek-R1-Distill-Llama-8B2026.06 | 7.2 | |
| Star1Backbone Model=DeepSeek-R1-Distill-Llama-70B2026.06 | 7.6 | |
| LLAMA 3 8B INSTRUCTBackbone=LLAMA 3 8B INSTRUCT2026.03 | 8.4 | |
| SafePathBackbone Model=DeepSeek-R1-Distill-Llama-70B2026.06 | 10 | |
| SurgicalBase Model=Gemma22026.04 | 10.4 | |
| AdaCDBase Model=Gemma22026.04 | 10.4 | |
| Low-Rank CombinationBackbone=LLAMA 3 8B INSTRUCT, Method=Low-Rank Combination2026.03 | 11.2 | |
| SelfCDBase Model=Gemma22026.04 | 11.2 | |
| PromptBase Model=Gemma22026.04 | 12 | |
| SafePathBackbone Model=DeepSeek-R1-Distill-Llama-8B2026.06 | 12 | |
| PromptBase Model=Llama32026.04 | 14.8 | |
| SSDBase Model=Gemma22026.04 | 16.4 | |
| DefaultBase Model=Gemma22026.04 | 17.6 | |
| Low-Rank CombinationBackbone=DEEPSEEK R1 DISTILL LLAMA, Method=Low-Rank Combination2026.03 | 23.6 | |
| REFUSE-LLAMABackbone=REFUSE-LLAMA2026.03 | 28 | |
| DEEPSEEK R1 DISTILL LLAMABackbone=DEEPSEEK R1 DISTILL LLAMA2026.03 | 28 | |
| SSDBase Model=Llama32026.04 | 49.6 |