Over-Refusal Evaluation on OrBench-Hard-1k, FalseReject, MorBench, and OverBench
23.22ORR (OrBench-Hard-1k)YuFeng-XGuard-0.6B
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| YuFeng-XGuard-0.6BInput Context=Prompt Only, Guardrail Category=Generative, Protected LLM=DeepSeek-R12026.06 | 23.22 | 10.72 | 3.08 | 5.46 | 10.62 | |
| FreoStreamGuardrail Category=Stream, Protected LLM=DeepSeek-R12026.06 | 24.44 | 5.36 | 4.68 | 2.99 | 9.37 | |
| FreoStreamGuardrail Category=Stream, Ablation=w/o Safety-Aligned Optimization, Protected LLM=DeepSeek-R12026.06 | 24.44 | 5.63 | 4.79 | 2.34 | 9.3 | |
| Qwen-Stream-8BGuardrail Category=Stream, Protected LLM=DeepSeek-R12026.06 | 28.31 | 6.18 | 6.97 | 3.26 | 11.27 | |
| Nemotron-4BGuardrail Category=Generative, Protected LLM=DeepSeek-R12026.06 | 30.1 | 12.6 | 17.45 | 5.58 | 16.43 | |
| YuFeng-XGuard-0.6BInput Context=Prompt+Response, Guardrail Category=Generative, Protected LLM=DeepSeek-R12026.06 | 31.57 | 9.65 | 5.02 | 5.46 | 12.93 | |
| Qwen-Gen-0.6BInput Context=Prompt+Response, Guardrail Category=Generative, Protected LLM=DeepSeek-R12026.06 | 37.68 | 10.19 | 10.95 | 3.9 | 15.68 | |
| GuardReasoner-1BGuardrail Category=Generative, Protected LLM=DeepSeek-R12026.06 | 43.58 | 7.51 | 11.52 | 4.38 | 16.75 | |
| Qwen-Stream-0.6BGuardrail Category=Stream, Protected LLM=DeepSeek-R12026.06 | 45.82 | 9.12 | 13.68 | 8.89 | 19.38 | |
| FreoStreamGuardrail Category=Stream, Ablation=w/o Future-Aware Reasoning, Protected LLM=DeepSeek-R12026.06 | 48.72 | 9.38 | 15.28 | 10.1 | 20.87 | |
| Qwen-Gen-0.6BInput Context=Prompt Only, Guardrail Category=Generative, Protected LLM=DeepSeek-R12026.06 | 53.36 | 10.46 | 19.27 | 5.2 | 22.07 |