Over-refusal Evaluation Suite

Benchmarks

TaskDatasetSOTA resultTrendResults

Over-refusal Evaluation Suite (XSTest, WildJailbreak, WildGuard, OKTest, OR-Bench)

3.2XSTest Refusal Rate (%)

Over-refusal Evaluation Suite OB, FB

9.9Overrefusal Rate (OB)