List your company
ResearchFormerly WizWand

Over-refusal Evaluation on XSTest and CoCoNot

2.6XSTest Refusal RateHARC
-1.29625.00251.377.598Jul 1, 2026
Updated 4mo ago

Evaluation results

MethodLinks
HARCBackbone=QWEN-2.5-7B2026.072.64.2
HARCBackbone=LLAMA-3.1-8B2026.073.58.1
Circuit BreakerBackbone=QWEN-2.5-7B2026.075.78.1
HARC + DPOBackbone=QWEN-2.5-7B2026.075.79.4
BaselineBackbone=QWEN-2.5-7B2026.079.19.4
HARC + DPOBackbone=LLAMA-3.1-8B2026.071019.5
BaselineBackbone=LLAMA-3.1-8B2026.0710.97.4
Circuit BreakerBackbone=LLAMA-3.1-8B2026.0711.35.4
CASTBackbone=QWEN-2.5-7B2026.0712.613.4
DPOBackbone=QWEN-2.5-7B2026.0716.515.4
SFTBackbone=LLAMA-3.1-8B2026.0723.927.5
CASTBackbone=LLAMA-3.1-8B2026.0728.326.2
SFTBackbone=QWEN-2.5-7B2026.0733.940.9
DPOBackbone=LLAMA-3.1-8B2026.0748.822.1
RepBendBackbone=LLAMA-3.1-8B2026.0787.880.5
RepBendBackbone=QWEN-2.5-7B2026.07100100