Safety Evaluation on Harmfulness Evaluation Sequences
0.79Harmfulness Scorellama2-13b-chat
Evaluation Results
| Method | Links | |
|---|---|---|
| llama2-13b-chatLearning Rate=3e-5, Attack Steps=1k2024.05 | 0.79 | |
| Qwen-4B-chatLearning Rate=8e-5, Attack Steps=1k2024.05 | 0.77 | |
| llama2-13b-chatLearning Rate=3e-5, Attack Steps=10k2024.05 | 0.76 | |
| Qwen-1.8B-chatLearning Rate=6e-5, Attack Steps=1k2024.05 | 0.76 | |
| Qwen-1.8B-chatLearning Rate=6e-5, Attack Steps=10k2024.05 | 0.76 | |
| Qwen-4B-chatLearning Rate=6e-5, Attack Steps=1k2024.05 | 0.76 | |
| Qwen-7B-chatLearning Rate=6e-5, Attack Steps=10k2024.05 | 0.76 | |
| Qwen-7B-chatLearning Rate=8e-5, Attack Steps=1k2024.05 | 0.76 | |
| Qwen-7B-chatLearning Rate=8e-5, Attack Steps=10k2024.05 | 0.76 | |
| llama2-13b-chatLearning Rate=8e-5, Attack Steps=1k2024.05 | 0.75 | |
| llama2-13b-chatLearning Rate=8e-5, Attack Steps=10k2024.05 | 0.75 | |
| Qwen-1.8B-chatLearning Rate=8e-5, Attack Steps=1k2024.05 | 0.75 | |
| Qwen-1.8B-chatLearning Rate=8e-5, Attack Steps=10k2024.05 | 0.75 | |
| Qwen-4B-chatLearning Rate=6e-5, Attack Steps=10k2024.05 | 0.75 | |
| Qwen-4B-chatLearning Rate=8e-5, Attack Steps=10k2024.05 | 0.75 | |
| llama2-7b-chatLearning Rate=3e-5, Attack Steps=10k2024.05 | 0.74 | |
| llama2-7b-chatLearning Rate=8e-5, Attack Steps=1k2024.05 | 0.74 | |
| Qwen-1.8B-chatLearning Rate=3e-5, Attack Steps=10k2024.05 | 0.74 | |
| Qwen-4B-chatLearning Rate=3e-5, Attack Steps=10k2024.05 | 0.74 | |
| Qwen-7B-chatLearning Rate=6e-5, Attack Steps=1k2024.05 | 0.74 | |
| llama2-7b-chatLearning Rate=6e-5, Attack Steps=1k2024.05 | 0.73 | |
| llama2-7b-chatLearning Rate=8e-5, Attack Steps=10k2024.05 | 0.73 | |
| llama2-13b-chatLearning Rate=6e-5, Attack Steps=1k2024.05 | 0.73 | |
| llama2-13b-chatLearning Rate=6e-5, Attack Steps=10k2024.05 | 0.73 | |
| Qwen-0.5B-chatLearning Rate=8e-5, Attack Steps=10k2024.05 | 0.73 | |
| llama2-7b-chatLearning Rate=6e-5, Attack Steps=10k2024.05 | 0.72 | |
| Qwen-1.8B-chatLearning Rate=3e-5, Attack Steps=1k2024.05 | 0.72 | |
| Qwen-4B-chatLearning Rate=3e-5, Attack Steps=1k2024.05 | 0.72 | |
| Qwen-7B-chatLearning Rate=3e-5, Attack Steps=10k2024.05 | 0.72 | |
| Qwen-0.5B-chatLearning Rate=6e-5, Attack Steps=10k2024.05 | 0.71 | |
| Qwen-0.5B-chatLearning Rate=8e-5, Attack Steps=1k2024.05 | 0.7 | |
| Qwen-7B-chatLearning Rate=3e-5, Attack Steps=1k2024.05 | 0.7 | |
| Qwen-0.5B-chatLearning Rate=6e-5, Attack Steps=1k2024.05 | 0.68 | |
| Qwen-0.5B-chatLearning Rate=3e-5, Attack Steps=10k2024.05 | 0.66 | |
| Qwen-0.5B-chatLearning Rate=3e-5, Attack Steps=1k2024.05 | 0.5 | |
| llama2-7b-chatLearning Rate=3e-5, Attack Steps=1k2024.05 | 0.47 | |
| RepNoise (llama2-7b-chat)Learning Rate=6e-5, Attack Steps=10k2024.05 | 0.13 | |
| RepNoise (llama2-7b-chat)Learning Rate=3e-5, Attack Steps=10k2024.05 | 0.12 | |
| RepNoise (llama2-7b-chat)Learning Rate=8e-5, Attack Steps=10k2024.05 | 0.12 | |
| RepNoise (Qwen-0.5B-chat)Learning Rate=8e-5, Attack Steps=1k2024.05 | 0.12 | |
| RepNoise (llama2-7b-chat)Learning Rate=8e-5, Attack Steps=1k2024.05 | 0.11 | |
| RepNoise (llama2-7b-chat)Learning Rate=6e-5, Attack Steps=1k2024.05 | 0.1 | |
| RepNoise (Qwen-0.5B-chat)Learning Rate=3e-5, Attack Steps=10k2024.05 | 0.1 | |
| RepNoise (Qwen-4B-chat)Learning Rate=8e-5, Attack Steps=10k2024.05 | 0.1 | |
| RepNoise (Qwen-0.5B-chat)Learning Rate=6e-5, Attack Steps=1k2024.05 | 0.09 | |
| RepNoise (Qwen-4B-chat)Condition=pre-attack2024.05 | 0.09 | |
| RepNoise (llama2-7b-chat)Learning Rate=3e-5, Attack Steps=1k2024.05 | 0.08 | |
| Qwen-7B-chatCondition=pre-attack2024.05 | 0.08 | |
| llama2-13b-chatCondition=pre-attack2024.05 | 0.07 | |
| RepNoise (Qwen-0.5B-chat)Learning Rate=3e-5, Attack Steps=1k2024.05 | 0.07 | |
| RepNoise (Qwen-0.5B-chat)Condition=pre-attack2024.05 | 0.06 | |
| llama2-7b-chatCondition=pre-attack2024.05 | 0.05 | |
| RepNoise (llama2-7b-chat)Condition=pre-attack2024.05 | 0.05 | |
| RepNoise (llama2-13b-chat)Learning Rate=6e-5, Attack Steps=10k2024.05 | 0.05 | |
| RepNoise (Qwen-4B-chat)Learning Rate=6e-5, Attack Steps=10k2024.05 | 0.04 | |
| RepNoise (Qwen-1.8B-chat)Condition=pre-attack2024.05 | 0.02 | |
| RepNoise (Qwen-4B-chat)Learning Rate=3e-5, Attack Steps=1k2024.05 | 0.02 | |
| RepNoise (Qwen-4B-chat)Learning Rate=6e-5, Attack Steps=1k2024.05 | 0.02 | |
| RepNoise (Qwen-0.5B-chat)Learning Rate=8e-5, Attack Steps=10k2024.05 | 0.01 | |
| RepNoise (Qwen-1.8B-chat)Learning Rate=3e-5, Attack Steps=1k2024.05 | 0.01 | |
| RepNoise (Qwen-7B-chat)Condition=pre-attack2024.05 | 0.01 | |
| RepNoise (Qwen-7B-chat)Learning Rate=3e-5, Attack Steps=10k2024.05 | 0.01 | |
| RepNoise (Qwen-7B-chat)Learning Rate=8e-5, Attack Steps=10k2024.05 | 0.01 | |
| RepNoise (llama2-13b-chat)Condition=pre-attack2024.05 | 0 | |
| RepNoise (llama2-13b-chat)Learning Rate=3e-5, Attack Steps=1k2024.05 | 0 | |
| RepNoise (llama2-13b-chat)Learning Rate=3e-5, Attack Steps=10k2024.05 | 0 | |
| RepNoise (llama2-13b-chat)Learning Rate=6e-5, Attack Steps=1k2024.05 | 0 | |
| RepNoise (llama2-13b-chat)Learning Rate=8e-5, Attack Steps=1k2024.05 | 0 | |
| RepNoise (llama2-13b-chat)Learning Rate=8e-5, Attack Steps=10k2024.05 | 0 | |
| Qwen-0.5B-chatCondition=pre-attack2024.05 | 0 | |
| RepNoise (Qwen-0.5B-chat)Learning Rate=6e-5, Attack Steps=10k2024.05 | 0 | |
| Qwen-1.8B-chatCondition=pre-attack2024.05 | 0 | |
| RepNoise (Qwen-1.8B-chat)Learning Rate=3e-5, Attack Steps=10k2024.05 | 0 | |
| RepNoise (Qwen-1.8B-chat)Learning Rate=6e-5, Attack Steps=1k2024.05 | 0 | |
| RepNoise (Qwen-1.8B-chat)Learning Rate=6e-5, Attack Steps=10k2024.05 | 0 | |
| RepNoise (Qwen-1.8B-chat)Learning Rate=8e-5, Attack Steps=1k2024.05 | 0 | |
| RepNoise (Qwen-1.8B-chat)Learning Rate=8e-5, Attack Steps=10k2024.05 | 0 | |
| Qwen-4B-chatCondition=pre-attack2024.05 | 0 | |
| RepNoise (Qwen-4B-chat)Learning Rate=3e-5, Attack Steps=10k2024.05 | 0 | |
| RepNoise (Qwen-4B-chat)Learning Rate=8e-5, Attack Steps=1k2024.05 | 0 | |
| RepNoise (Qwen-7B-chat)Learning Rate=3e-5, Attack Steps=1k2024.05 | 0 | |
| RepNoise (Qwen-7B-chat)Learning Rate=6e-5, Attack Steps=1k2024.05 | 0 | |
| RepNoise (Qwen-7B-chat)Learning Rate=6e-5, Attack Steps=10k2024.05 | 0 | |
| RepNoise (Qwen-7B-chat)Learning Rate=8e-5, Attack Steps=1k2024.05 | 0 |