Safety on Beavertails
1.2Violation RateVanilla SFT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Vanilla SFTAlignment Dataset=DirectRefusal, Backbone=s1.1-7B2026.02 | 1.2 | — | |
| DGR SFTAlignment Dataset=DirectRefusal, Backbone=s1.1-7B2026.02 | 1.9 | — | |
| DGR SFTAlignment Dataset=STAR-1, Backbone=s1.1-7B2026.02 | 4.8 | — | |
| Vanilla SFTAlignment Dataset=STAR-1, Backbone=s1.1-7B2026.02 | 17.1 | — | |
| Qwen2.5-7B-Instruct2026.02 | 18.6 | — | |
| DGR SFTAlignment Dataset=R1-ACT, Backbone=s1.1-7B2026.02 | 39.9 | — | |
| Vanilla SFTAlignment Dataset=R1-ACT, Backbone=s1.1-7B2026.02 | 40 | — | |
| RACOw_help=0.2, Backbone=Qwen3-4B-Base, Comparison Baseline=AMoPO2026.02 | 43.37 | — | |
| RACOw_help=0.2, Backbone=Gemma3-4B-Instruct, Comparison Baseline=AMoPO2026.02 | 45.78 | — | |
| RACOw_help=0.5, Backbone=Gemma3-4B-Instruct, Comparison Baseline=AMoPO2026.02 | 47 | — | |
| RACOw_help=0.2, Backbone=Gemma3-4B-Instruct, Comparison Baseline=DPO LW2026.02 | 47.59 | — | |
| RACOw_help=0.5, Backbone=Gemma3-4B-Instruct, Comparison Baseline=DPO LW2026.02 | 48.8 | — | |
| RACOw_help=0.5, Backbone=Qwen3-4B-Base, Comparison Baseline=AMoPO2026.02 | 49.39 | — | |
| RACOw_help=0.5, Backbone=Qwen3-4B-Instruct, Comparison Baseline=DPO LW2026.02 | 50.6 | — | |
| RACOw_help=0.8, Backbone=Qwen3-4B-Instruct, Comparison Baseline=DPO LW2026.02 | 53.01 | — | |
| RACOw_help=0.8, Backbone=Qwen3-4B-Base, Comparison Baseline=DPO LW2026.02 | 53.61 | — | |
| RACOw_help=0.8, Backbone=Qwen3-4B-Base, Comparison Baseline=AMoPO2026.02 | 54.21 | — | |
| RACOw_help=0.8, Backbone=Gemma3-4B-Instruct, Comparison Baseline=DPO LW2026.02 | 54.82 | — | |
| s1.1-7B2026.02 | 55.1 | — | |
| RACOw_help=0.2, Backbone=Qwen3-4B-Base, Comparison Baseline=DPO LW2026.02 | 57.83 | — | |
| RACOw_help=0.2, Backbone=Gemma3-4B-Base, Comparison Baseline=DPO LW2026.02 | 57.83 | — | |
| RACOw_help=0.5, Backbone=Qwen3-4B-Instruct, Comparison Baseline=AMoPO2026.02 | 59.03 | — | |
| RACOw_help=0.5, Backbone=Gemma3-4B-Base, Comparison Baseline=DPO LW2026.02 | 59.04 | — | |
| RACOw_help=0.2, Backbone=Qwen3-4B-Instruct, Comparison Baseline=DPO LW2026.02 | 63.86 | — | |
| RACOw_help=0.8, Backbone=Gemma3-4B-Instruct, Comparison Baseline=AMoPO2026.02 | 63.86 | — | |
| RACOw_help=0.5, Backbone=Qwen3-4B-Base, Comparison Baseline=DPO LW2026.02 | 64.46 | — | |
| RACOw_help=0.8, Backbone=Qwen3-4B-Instruct, Comparison Baseline=AMoPO2026.02 | 66.27 | — | |
| RACOw_help=0.2, Backbone=Gemma3-4B-Base, Comparison Baseline=AMoPO2026.02 | 66.27 | — | |
| RACOw_help=0.8, Backbone=Gemma3-4B-Base, Comparison Baseline=DPO LW2026.02 | 70.48 | — | |
| RACOw_help=0.5, Backbone=Gemma3-4B-Base, Comparison Baseline=AMoPO2026.02 | 71.08 | — | |
| RACOw_help=0.8, Backbone=Gemma3-4B-Base, Comparison Baseline=AMoPO2026.02 | 76.51 | — | |
| RACOw_help=0.2, Backbone=Qwen3-4B-Instruct, Comparison Baseline=AMoPO2026.02 | 78.92 | — | |
| DPO_WhispererAlignment=DPO, Data Recipe=ear-whisperer, Base Model=SFT_DB2025.05 | — | 97.04 | |
| Mixtral (Base)LLM=Mixtral, SFT Status=None2025.05 | — | 76 | |
| Mixtral (SFT_DB)LLM=Mixtral, SFT Status=SFT on AIDSAFE-generated CoTs and responses2025.05 | — | 96 | |
| Mixtral (SFT_OG)LLM=Mixtral, SFT Status=SFT on original response data without CoTs2025.05 | — | 79.57 | |
| Qwen (Base)LLM=Qwen, SFT Status=None2025.05 | — | 94.14 | |
| Qwen (SFT_DB)LLM=Qwen, SFT Status=SFT on AIDSAFE-generated CoTs and responses2025.05 | — | 97 | |
| Qwen (SFT_OG)LLM=Qwen, SFT Status=SFT on original response data without CoTs2025.05 | — | 87.95 | |
| SFT_DBTraining=Supervised Fine-Tuning, Data Generation=AIDSAFE2025.05 | — | 96 |