Over-refusal evaluation on OR-Bench-Hard-1K (test)
60.6F1 ScoreRobust Harmful Features detector
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Robust Harmful Features detectorBase Model=Llama-3-8B2026.06 | 60.6 | 41.8 | |
| VanillaBase Model=Llama-3-8B2026.06 | 57.1 | 32.9 | |
| GradientCuffBase Model=Llama-3-8B2026.06 | 51 | 5.3 | |
| CASTBase Model=Llama-3-8B2026.06 | 45.3 | 29 |