Over-Refusal Evaluation on FalseReject
9.12ORRFreoStream
Evaluation results
| Method | Links | |
|---|---|---|
| FreoStreamGuardrail Type=Stream, Protected LLM=GPT-4o, Future-Aware Reasoning=true, Safety-Aligned Optimization=true2026.06 | 9.12 | |
| w/o Safety-Aligned OptimizationGuardrail Type=Stream, Protected LLM=GPT-4o, Future-Aware Reasoning=true, Safety-Aligned Optimization=false2026.06 | 9.73 | |
| YuFeng-XGuard-0.6B Prompt+ResponseGuardrail Type=Generative, Input Context=Prompt+Response, Protected LLM=GPT-4o2026.06 | 9.74 | |
| Qwen-Stream-8BGuardrail Type=Stream, Protected LLM=GPT-4o2026.06 | 10.83 | |
| Qwen-Gen-0.6B Prompt+ResponseGuardrail Type=Generative, Input Context=Prompt+Response, Protected LLM=GPT-4o2026.06 | 14 | |
| YuFeng-XGuard-0.6B Prompt OnlyGuardrail Type=Generative, Input Context=Prompt Only, Protected LLM=GPT-4o2026.06 | 16.69 | |
| Qwen-Stream-0.6BGuardrail Type=Stream, Protected LLM=GPT-4o2026.06 | 17.64 | |
| w/o Future-Aware ReasoningGuardrail Type=Stream, Protected LLM=GPT-4o, Future-Aware Reasoning=false, Safety-Aligned Optimization=true2026.06 | 18.49 | |
| Qwen-Gen-0.6B Prompt OnlyGuardrail Type=Generative, Input Context=Prompt Only, Protected LLM=GPT-4o2026.06 | 20.92 | |
| Nemotron-4BGuardrail Type=Generative, Protected LLM=GPT-4o2026.06 | 21.44 | |
| GuardReasoner-1BGuardrail Type=Generative, Protected LLM=GPT-4o2026.06 | 27.41 |