Over-refusal Evaluation on OR-Bench (boundary cases)
1.7OR-FPRMistral-v2-7B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Mistral-v2-7BBackbone=Mistral-v2-7B2026.05 | 1.7 | 56.4 | |
| Qwen-2.5-7BBackbone=Qwen-2.5-7B2026.05 | 4.6 | 74.8 | |
| LLaMA-3-8BBackbone=LLaMA-3-8B2026.05 | 7.4 | 56.5 | |
| bi-level adversarial training framework (Ours)Backbone=LLaMA-3-8B2026.05 | 16.6 | 46.5 | |
| bi-level adversarial training framework (Ours)Backbone=Mistral-v2-7B2026.05 | 20.7 | 36.6 | |
| bi-level adversarial training framework (Ours)Backbone=Qwen-2.5-7B2026.05 | 23.3 | 53.5 | |
| AlphaSteerBackbone=LLaMA-3-8B2026.05 | 24.4 | 37.9 | |
| AlphaSteerBackbone=Qwen-2.5-7B2026.05 | 25.5 | 49.8 | |
| AlphaSteerBackbone=Mistral-v2-7B2026.05 | 25.6 | 31.4 | |
| Latent Adversarial Training (LAT)Backbone=Qwen-2.5-7B2026.05 | 28.3 | 46.8 | |
| ROSIBackbone=LLaMA-3-8B2026.05 | 31.4 | 30.1 | |
| ROSIBackbone=Qwen-2.5-7B2026.05 | 31.8 | 42.2 | |
| Latent Adversarial Training (LAT)Backbone=Mistral-v2-7B2026.05 | 32.3 | 24.5 | |
| Circuit-Breaker (CB)Backbone=Qwen-2.5-7B2026.05 | 35.5 | 40.3 | |
| ROSIBackbone=Mistral-v2-7B2026.05 | 35.7 | 20.7 | |
| Latent Adversarial Training (LAT)Backbone=LLaMA-3-8B2026.05 | 38.4 | 23.5 | |
| Circuit-Breaker (CB)Backbone=LLaMA-3-8B2026.05 | 49 | 14 | |
| Circuit-Breaker (CB)Backbone=Mistral-v2-7B2026.05 | 83.3 | -26 |