Over-refusal Evaluation on Over-refusal (test)
0Refusal RateNo train
Evaluation Results
| Method | Links | |
|---|---|---|
| No trainBackbone=R1-7B, Dataset Size=-, Temperature=0.02026.04 | 0 | |
| SafeChainBackbone=R1-14B, Dataset Size=40K, Temperature=0.02026.04 | 0 | |
| SafeChainBackbone=R1-14B, Dataset Size=1K, Temperature=0.02026.04 | 0 | |
| No trainBackbone=R1-32B, Dataset Size=-, Temperature=0.02026.04 | 0 | |
| SafeChainBackbone=R1-32B, Dataset Size=1K, Temperature=0.02026.04 | 0 | |
| No trainBackbone=s1-3B, Dataset Size=-, Temperature=0.02026.04 | 0 | |
| No trainBackbone=s1-14B, Dataset Size=-, Temperature=0.02026.04 | 0 | |
| SafeChainBackbone=R1-1.5B, Dataset Size=40k, Temperature=0.02026.04 | 0.4 | |
| SafeChainBackbone=R1-7B, Dataset Size=1K, Temperature=0.02026.04 | 0.4 | |
| No trainBackbone=R1-8B, Dataset Size=-, Temperature=0.02026.04 | 0.4 | |
| SafeChainBackbone=R1-8B, Dataset Size=40K, Temperature=0.02026.04 | 0.4 | |
| SafeChainBackbone=R1-32B, Dataset Size=40K, Temperature=0.02026.04 | 0.4 | |
| SafeChainBackbone=s1-3B, Dataset Size=1K, Temperature=0.02026.04 | 0.4 | |
| SafeChainBackbone=s1-14B, Dataset Size=1K, Temperature=0.02026.04 | 0.4 | |
| SafeChainBackbone=R1-7B, Dataset Size=40K, Temperature=0.02026.04 | 0.8 | |
| SafeChainBackbone=R1-8B, Dataset Size=1K, Temperature=0.02026.04 | 0.8 | |
| No trainBackbone=R1-14B, Dataset Size=-, Temperature=0.02026.04 | 0.8 | |
| No trainBackbone=R1-1.5B, Dataset Size=-, Temperature=0.02026.04 | 2 | |
| SafeChainBackbone=R1-1.5B, Dataset Size=1K, Temperature=0.02026.04 | 2.8 | |
| S1-ALTBackbone=s1-3B, Dataset Size=1K, Temperature=0.02026.04 | 6 | |
| STAR-1Backbone=R1-32B, Dataset Size=1K, Temperature=0.02026.04 | 9.6 | |
| STAR-1Backbone=R1-14B, Dataset Size=1K, Temperature=0.02026.04 | 10.4 | |
| R1-ALTBackbone=R1-32B, Dataset Size=1K, Temperature=0.02026.04 | 11.2 | |
| R1-ALTBackbone=R1-8B, Dataset Size=1K, Temperature=0.02026.04 | 14 | |
| S1-ALTBackbone=s1-14B, Dataset Size=1K, Temperature=0.02026.04 | 14 | |
| DirectRefusalBackbone=R1-14B, Dataset Size=1k, Temperature=0.02026.04 | 14.8 | |
| DirectRefusalBackbone=R1-32B, Dataset Size=1k, Temperature=0.02026.04 | 15.6 | |
| DirectRefusalBackbone=R1-8B, Dataset Size=1k, Temperature=0.02026.04 | 18.8 | |
| R1-ALTBackbone=R1-14B, Dataset Size=1K, Temperature=0.02026.04 | 20.8 | |
| STAR-1Backbone=R1-8B, Dataset Size=1K, Temperature=0.02026.04 | 21.2 | |
| DirectRefusalBackbone=R1-7B, Dataset Size=1k, Temperature=0.02026.04 | 29.2 | |
| STAR-1Backbone=R1-7B, Dataset Size=1K, Temperature=0.02026.04 | 30 | |
| R1-ALTBackbone=R1-7B, Dataset Size=1K, Temperature=0.02026.04 | 31.6 | |
| STAR-1Backbone=R1-1.5B, Dataset Size=1K, Temperature=0.02026.04 | 64 | |
| R1-ALTBackbone=R1-1.5B, Dataset Size=1K, Temperature=0.02026.04 | 69.6 | |
| DirectRefusalBackbone=R1-1.5B, Dataset Size=1k, Temperature=0.02026.04 | 72.4 |