Safety-aware fine-tuning on DAHOAS RM-HH-RLHF (val)
0.43Evaluation LossOnline dynamic
Evaluation results
| Method | Links | |
|---|---|---|
| Online dynamicsample ratio (R)=10%, G=1, backbone=LLAMA-8B2025.11 | 0.43 | |
| Onlinesample ratio (R)=10%, G=1, backbone=LLAMA-8B2025.11 | 0.47 | |
| Onlinesample ratio (R)=5%, G=1, backbone=LLAMA-8B2025.11 | 0.49 | |
| Offline BDSbackbone=LLAMA-8B2025.11 | 0.53 | |
| Direct mixingmixing ratio (ρ)=0.5, backbone=LLAMA-8B2025.11 | 0.58 | |
| LESSbackbone=LLAMA-8B2025.11 | 0.64 | |
| Direct mixingmixing ratio (ρ)=1, backbone=LLAMA-8B2025.11 | 0.71 | |
| Random selectionbackbone=LLAMA-8B2025.11 | 0.74 | |
| DSIRbackbone=LLAMA-8B2025.11 | 0.82 |