List your company
ResearchFormerly WizWand

Overrefusal Evaluation on OrBench-H

99.85RRDb as Alpaca
1.133226.761652.3978.0184Oct 15, 2025Nov 23, 2025Jan 2, 2026Feb 11, 2026Mar 23, 2026May 2, 2026Jun 11, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Db as AlpacaSetting / Model=P-SFT, Llama22026.0399.85
Db as AlpacaSetting / Model=RLVR, Llama3-U2026.0398.48
AutoSteerBackbone=LLaVA-1.5-7B2025.1089.3
DEEPSEEK R1 DISTILL LLAMABackbone=DEEPSEEK R1 DISTILL LLAMA2026.0384.61
Low-Rank CombinationBackbone=DEEPSEEK R1 DISTILL LLAMA, Method=Low-Rank Combination2026.0377.1
FigStepBackbone=LLaVA-1.5-7B2025.1072.6
LLAMA 3 8B INSTRUCTBackbone=LLAMA 3 8B INSTRUCT2026.0360.58
Low-Rank CombinationBackbone=LLAMA 3 8B INSTRUCT, Method=Low-Rank Combination2026.0357.92
Db as Our DataSetting / Model=RLVR, Llama3-U2026.0357.09
Db as Our DataSetting / Model=P-SFT, Llama22026.0355.34
Nemotron-4BGuardrail Type=Generative, Protected LLM=GPT-4o2026.0653
w/o Future-Aware ReasoningGuardrail Type=Stream, Protected LLM=GPT-4o, Future-Aware Reasoning=false, Safety-Aligned Optimization=true2026.0650.42
Qwen-Stream-0.6BGuardrail Type=Stream, Protected LLM=GPT-4o2026.0649.86
Qwen-Gen-0.6B Prompt OnlyGuardrail Type=Generative, Input Context=Prompt Only, Protected LLM=GPT-4o2026.0647.77
CoCABackbone=LLaVA-1.5-7B2025.1037.5
ETABackbone=LLaVA-1.5-7B2025.1035.9
Qwen-Gen-0.6B Prompt+ResponseGuardrail Type=Generative, Input Context=Prompt+Response, Protected LLM=GPT-4o2026.0631.37
MoRASBackbone=LLaVA-1.5-7B2025.1027.7
ECSOBackbone=LLaVA-1.5-7B2025.1025.2
ASTRABackbone=LLaVA-1.5-7B2025.1024.2
REFUSE-LLAMABackbone=REFUSE-LLAMA2026.0323.88
VanillaBackbone=LLaVA-1.5-7B2025.1023.4
YuFeng-XGuard-0.6B Prompt OnlyGuardrail Type=Generative, Input Context=Prompt Only, Protected LLM=GPT-4o2026.0622.81
Qwen-Stream-8BGuardrail Type=Stream, Protected LLM=GPT-4o2026.0622.66
BaselineSetting / Model=RLVR, Llama3-U2026.0316.83
YuFeng-XGuard-0.6B Prompt+ResponseGuardrail Type=Generative, Input Context=Prompt+Response, Protected LLM=GPT-4o2026.0615.74
Low-Rank CombinationBackbone=REFUSE-LLAMA, Method=Low-Rank Combination2026.0312.89
w/o Safety-Aligned OptimizationGuardrail Type=Stream, Protected LLM=GPT-4o, Future-Aware Reasoning=true, Safety-Aligned Optimization=false2026.0611.66
FreoStreamGuardrail Type=Stream, Protected LLM=GPT-4o, Future-Aware Reasoning=true, Safety-Aligned Optimization=true2026.0611.56
GuardReasoner-1BGuardrail Type=Generative, Protected LLM=GPT-4o2026.0610.17
Categorical SteeringBackbone=REFUSE-LLAMA, Method=Categorical Steering2026.035.84
BaselineSetting / Model=P-SFT, Llama22026.034.93