Over-refusal Rate Analysis on OR-Bench
16.68Over-refusal RateAdaCD
Evaluation results
| Method | Links | |
|---|---|---|
| AdaCDBase Model=Qwen32026.04 | 16.68 | |
| SelfCDBase Model=Qwen32026.04 | 18.2 | |
| SurgicalBase Model=Qwen32026.04 | 22.52 | |
| SSPModel Backbone=Mistral3-8B2026.01 | 24.1 | |
| SSPModel Backbone=LLaMA3-8B2026.01 | 24.6 | |
| SSPModel Backbone=Qwen2.5-7B2026.01 | 25.3 | |
| ACE-SafetyModel Backbone=Mistral3-8B2026.01 | 28 | |
| SafeDecodingModel Backbone=Mistral3-8B2026.01 | 28.7 | |
| ACE-SafetyModel Backbone=LLaMA3-8B2026.01 | 28.7 | |
| SafeDecodingModel Backbone=LLaMA3-8B2026.01 | 29.3 | |
| ACE-SafetyModel Backbone=Qwen2.5-7B2026.01 | 29.6 | |
| SafeDecodingModel Backbone=Qwen2.5-7B2026.01 | 30.1 | |
| DefaultBase Model=Qwen32026.04 | 30.63 | |
| AdaCDBase Model=Llama32026.04 | 30.93 | |
| SmoothLLMModel Backbone=Mistral3-8B2026.01 | 32.6 | |
| SmoothLLMModel Backbone=LLaMA3-8B2026.01 | 33.2 | |
| Self-ReminderModel Backbone=Mistral3-8B2026.01 | 34.5 | |
| SmoothLLMModel Backbone=Qwen2.5-7B2026.01 | 34.5 | |
| SSDBase Model=Qwen32026.04 | 35.03 | |
| Self-ReminderModel Backbone=LLaMA3-8B2026.01 | 35.1 | |
| Self-ReminderModel Backbone=Qwen2.5-7B2026.01 | 36.2 | |
| SelfCDBase Model=Llama32026.04 | 37.15 | |
| SurgicalBase Model=Llama32026.04 | 41.32 | |
| PromptBase Model=Qwen32026.04 | 53.75 | |
| SurgicalBase Model=Gemma22026.04 | 60.5 | |
| AdaCDBase Model=Gemma22026.04 | 62.55 | |
| DefaultBase Model=Llama32026.04 | 67.17 | |
| SelfCDBase Model=Gemma22026.04 | 69.14 | |
| PromptBase Model=Gemma22026.04 | 74.6 | |
| SSDBase Model=Gemma22026.04 | 75.97 | |
| DefaultBase Model=Gemma22026.04 | 78.7 | |
| PromptBase Model=Llama32026.04 | 92.95 | |
| SSDBase Model=Llama32026.04 | 99.32 |