Overrefusal Detection on OR-Bench
93.42AUROCT3+OCSVM
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| T3+OCSVMaugmented=false2026.02 | 93.42 | 25.17 | 96.62 | 92.93 | |
| DUOGUARD2026.02 | 93.11 | 43.5 | 97.29 | 90.63 | |
| T3+GMMaugmented=false2026.02 | 91.08 | 22.17 | 92.65 | 94.05 | |
| LLAMAGUARD3-1Bparameters=1B2026.02 | 89.05 | 14.83 | 92.4 | 93.46 | |
| POLYGUARD2026.02 | 87.17 | 53.17 | 91.81 | 88.33 | |
| T3+GMMaugmented=true, augmentation_source=GPT-OSS-20B2026.02 | 85.94 | 32.67 | 90.22 | 91.53 | |
| T3+OCSVMaugmented=true, augmentation_source=GPT-OSS-20B2026.02 | 85.81 | 41 | 91.14 | 90.6 | |
| MDJUDGE2026.02 | 85.77 | 89 | 94.78 | 90.82 | |
| LLAMAGUARD4-12Bparameters=12B2026.02 | 84.98 | 27.48 | 98.39 | 97.96 | |
| CIDER2026.02 | 79 | 71.17 | 90.19 | 85.36 | |
| GMM2026.02 | 75.3 | 68.83 | 86.79 | 85.47 | |
| RMD2026.02 | 74.74 | 75.5 | 86.74 | 84.91 | |
| VIM2026.02 | 71.62 | 72.5 | 83.86 | 84.93 | |
| NNGUIDE2026.02 | 62.2 | 92.83 | 78.62 | 83.33 | |
| ADASCALE2026.02 | 55.09 | 97.83 | 74.16 | 83.33 | |
| REACT2026.02 | 54.38 | 94.17 | 74.2 | 83.33 | |
| OPENMAX2026.02 | 47.1 | 98.17 | 68.71 | 83.33 | |
| FDBD2026.02 | 41.69 | 95.17 | 66.62 | 83.33 |