Over-refusal evaluation
Benchmarks
DatasetSOTA methodMetricTrendResultsLast updated
0ORR
72
Feb 26, 2026
16ORR
72
Feb 26, 2026
100Evaluation Score (avg@4)
70
Jul 3, 2026
1.6Over-refusal Rate
45
Jun 16, 2026
0Refusal Rate
36
Apr 23, 2026
37Math Score
32
Feb 26, 2026
0Biology Refusal Count
32
Feb 26, 2026
35.7Informative Refusal Rate
27
Jun 30, 2026
1.7OR-FPR
18
May 26, 2026
2.6XSTest Refusal Rate
16
Jul 2, 2026
23.22ORR (OrBench-Hard-1k)
11
Jun 15, 2026
5.59ORR
11
Jun 15, 2026
1.47ORR
11
Jun 15, 2026
9.12ORR
11
Jun 15, 2026
22OR Score
7
Jul 3, 2026
1.06Over-refusal Rate
7
Mar 17, 2026
1.58Over-refusal Rate
7
Mar 17, 2026
3.5Over-Refusal Rate
5
Jun 16, 2026
60.6F1 Score
4
Jun 29, 2026
17Over-Refusal Rate
4
Apr 20, 2026
0.035Over-refusal Rate
4
Feb 26, 2026