Safety Assessment on Safety Prompts (randomly selected 200 samples per field)
1.5Insensitivity Scorellama2 -> CP -> FT + 0.5 chat vector
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| llama2 -> CP -> FT + 0.5 chat vectorModel Base=Chinese-LLaMA 13B, Chat Vector Coefficient=0.52023.10 | 1.5 | 0 | 0.5 | 0.5 | 0 | 0.5 | 0 | |
| llama2 -> CP -> FT + chat vectorModel Base=Chinese-LLaMA 13B, Chat Vector Coefficient=1.02023.10 | 2 | 0.5 | 1 | 0.5 | 0 | 0 | 0 | |
| llama2 -> CP + chat vectorModel Base=Chinese-LLaMA 13B, Chat Vector Coefficient=1.02023.10 | 2.5 | 0 | 0.5 | 0.5 | 0 | 1 | 0.5 | |
| llama2 -> CP -> FTModel Base=Chinese-LLaMA 13B2023.10 | 5 | 2 | 3 | 1 | 0 | 1 | 0 | |
| llama2 -> CP -> FT + chat vectorModel Base=Traditional Chinese LLaMA 13B, Chat Vector Coefficient=1.02023.10 | 7.5 | 4 | 2.5 | 2 | 0 | 6 | 1.5 | |
| llama2-chat -> CP -> FTModel Base=Traditional Chinese LLaMA 13B2023.10 | 13 | 11.5 | 14.5 | 2.5 | 0 | 11 | 2.5 | |
| llama2 -> CP + chat vectorModel Base=Traditional Chinese LLaMA 13B, Chat Vector Coefficient=1.02023.10 | 13.5 | 3 | 8 | 5.5 | 1.5 | 6.5 | 5 | |
| llama2 -> CP + 0.5 chat vectorModel Base=Chinese-LLaMA 13B, Chat Vector Coefficient=0.52023.10 | 47.5 | 28.5 | 17 | 1 | 6 | 4.5 | 9 | |
| llama2 -> CP -> FTModel Base=Traditional Chinese LLaMA 13B2023.10 | 66 | 24 | 37.5 | 1.5 | 1 | 15.5 | 4 |