Unsafety evaluation on CATQA
37.7Chinese Safety ScoreSFT+DARE
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| SFT+DAREFine-tuning Protocol=PEFT2024.02 | 37.7 | 8.97 | 31.98 | 48.39 | 30.47 | 31.5 | 29.32 | |
| SFTFine-tuning Protocol=PEFT2024.02 | 36.03 | 21.16 | 33.21 | 46.39 | 31.07 | 33.57 | 31.39 | |
| SFT+DAREFine-tuning Protocol=Full-FT2024.02 | 28.88 | 4.73 | 36.91 | 25.9 | 13.66 | 22.02 | 19.84 | |
| SFTFine-tuning Protocol=Full-FT2024.02 | 28.87 | 8.04 | 33.98 | 26.98 | 12.91 | 22.16 | 19.98 | |
| SFT+RESTAFine-tuning Protocol=PEFT2024.02 | 14.98 | 6.2 | 13.54 | 20.93 | 10.46 | 13.22 | 11.04 | |
| SFT+RESTAdFine-tuning Protocol=PEFT2024.02 | 12.73 | 4.57 | 12.27 | 20.15 | 11.15 | 12.17 | 9.99 | |
| SFT+RESTAFine-tuning Protocol=Full-FT2024.02 | 7.09 | 1.27 | 9.38 | 3.85 | 1.82 | 4.68 | 2.5 | |
| SFT+RESTAdFine-tuning Protocol=Full-FT2024.02 | 4.75 | 1.27 | 10.17 | 3.67 | 1.82 | 4.34 | 2.16 |