Safety Evaluation on HARMFULQA (Domain Breakdown)
19.17Safety Score (Chinese)SFT
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| SFTFine-tuning Protocol=PEFT2024.02 | 19.17 | 6.15 | 19.48 | 29.74 | 16.49 | 18.21 | 16.68 | |
| SFT+DAREFine-tuning Protocol=PEFT2024.02 | 15.46 | 4.59 | 21.02 | 32.3 | 14.35 | 17.54 | 16.01 | |
| SFT+DAREFine-tuning Protocol=Full-FT2024.02 | 13.33 | 4.08 | 13.33 | 11.34 | 8.16 | 10.05 | 8.52 | |
| SFTFine-tuning Protocol=Full-FT2024.02 | 12.37 | 5.61 | 16.02 | 13.4 | 5.61 | 10.6 | 9.07 | |
| SFT+RESTAFine-tuning Protocol=PEFT2024.02 | 8.29 | 2.56 | 7.69 | 8.67 | 2.05 | 5.85 | 4.32 | |
| SFT+RESTAdFine-tuning Protocol=PEFT2024.02 | 7.21 | 3.07 | 5.64 | 7.69 | 3.61 | 5.44 | 3.91 | |
| SFT+RESTAdFine-tuning Protocol=Full-FT2024.02 | 3.09 | 0.51 | 4.1 | 2.58 | 1.02 | 2.16 | 0.63 | |
| SFT+RESTAFine-tuning Protocol=Full-FT2024.02 | 1.54 | 0.51 | 3.08 | 1.54 | 1.02 | 1.54 | 0.01 |