Harmfulness Evaluation on DirectHarm (test)
5Harmfulness Score (Llama-Guard-3B)Original
Evaluation results
| Method | Links | ||
|---|---|---|---|
| OriginalModel=Llama-2-7B-Chat, Utility Context=GSM8K2025.03 | 5 | 4.5 | |
| OriginalModel=Llama-2-7B-Chat, Utility Context=PubMedQA2025.03 | 5 | 4.5 | |
| RESTAModel=Llama-2-7B-Chat, Utility Context=PubMedQA2025.03 | 5.8 | 5.5 | |
| SafeInstructModel=Llama-2-7B-Chat, Utility Context=GSM8K2025.03 | 7.5 | 7.3 | |
| RESTAModel=Llama-2-7B-Chat, Utility Context=GSM8K2025.03 | 7.5 | 7.2 | |
| SafeMERGEModel=Llama-2-7B-Chat, Utility Context=GSM8K2025.03 | 7.5 | 7.1 | |
| RESTA-InstructModel=Llama-2-7B-Chat, Utility Context=PubMedQA2025.03 | 8.1 | 7.8 | |
| SafeMERGEModel=Llama-2-7B-Chat, Utility Context=PubMedQA2025.03 | 8.1 | 7.8 | |
| SafeMERGEModel=Qwen-2-7B-Instruct, Utility Context=GSM8K2025.03 | 8.2 | 7.8 | |
| SafeMERGEModel=Qwen-2-7B-Instruct, Utility Context=PubMedQA2025.03 | 8.5 | 7.9 | |
| SafeMERGEModel=Llama-3.1-8B-Instruct, Utility Context=GSM8K2025.03 | 8.8 | 8.3 | |
| SafeMERGEModel=Llama-3.1-8B-Instruct, Utility Context=PubMedQA2025.03 | 9.1 | 8.8 | |
| RESTA-InstructModel=Llama-2-7B-Chat, Utility Context=GSM8K2025.03 | 9.5 | 8.9 | |
| SafeMERGEModel=Qwen-2.5-7B-Instruct, Utility Context=GSM8K2025.03 | 10.1 | 9.6 | |
| SafeLoRAModel=Llama-2-7B-Chat, Utility Context=GSM8K2025.03 | 10.2 | 9.6 | |
| RESTAModel=Llama-3.1-8B-Instruct, Utility Context=PubMedQA2025.03 | 10.3 | 9.8 | |
| SafeLoRAModel=Llama-2-7B-Chat, Utility Context=PubMedQA2025.03 | 10.7 | 10.3 | |
| SafeMERGEModel=Qwen-2.5-7B-Instruct, Utility Context=PubMedQA2025.03 | 11.1 | 10.6 | |
| OriginalModel=Llama-3.1-8B-Instruct, Utility Context=GSM8K2025.03 | 11.3 | 10.7 | |
| OriginalModel=Llama-3.1-8B-Instruct, Utility Context=PubMedQA2025.03 | 11.3 | 10.7 | |
| SafeInstructModel=Llama-3.1-8B-Instruct, Utility Context=PubMedQA2025.03 | 11.8 | 11.3 | |
| RESTAModel=Llama-3.1-8B-Instruct, Utility Context=GSM8K2025.03 | 11.9 | 11.6 | |
| SafeInstructModel=Llama-2-7B-Chat, Utility Context=PubMedQA2025.03 | 12.2 | 11.8 | |
| Fine-tunedModel=Llama-2-7B-Chat, Utility Context=PubMedQA2025.03 | 12.5 | 11.9 | |
| SafeInstructModel=Llama-3.1-8B-Instruct, Utility Context=GSM8K2025.03 | 12.5 | 11.9 | |
| SafeInstructModel=Qwen-2-7B-Instruct, Utility Context=PubMedQA2025.03 | 12.5 | 12.1 | |
| SafeInstructModel=Qwen-2.5-7B-Instruct, Utility Context=GSM8K2025.03 | 12.5 | 11.8 | |
| SafeInstructModel=Qwen-2.5-7B-Instruct, Utility Context=PubMedQA2025.03 | 13.1 | 12.6 | |
| RESTA-InstructModel=Llama-3.1-8B-Instruct, Utility Context=GSM8K2025.03 | 13.5 | 12.9 | |
| SafeInstructModel=Qwen-2-7B-Instruct, Utility Context=GSM8K2025.03 | 13.7 | 13.4 | |
| RESTA-InstructModel=Llama-3.1-8B-Instruct, Utility Context=PubMedQA2025.03 | 14.2 | 13.8 | |
| OriginalModel=Qwen-2.5-7B-Instruct, Utility Context=GSM8K2025.03 | 14.2 | 13.7 | |
| OriginalModel=Qwen-2.5-7B-Instruct, Utility Context=PubMedQA2025.03 | 14.2 | 13.6 | |
| RESTA-InstructModel=Qwen-2.5-7B-Instruct, Utility Context=PubMedQA2025.03 | 14.9 | 14.1 | |
| SafeLoRAModel=Llama-3.1-8B-Instruct, Utility Context=GSM8K2025.03 | 15.1 | 14.5 | |
| RESTAModel=Qwen-2.5-7B-Instruct, Utility Context=PubMedQA2025.03 | 15.7 | 14.4 | |
| SafeLoRAModel=Llama-3.1-8B-Instruct, Utility Context=PubMedQA2025.03 | 16.7 | 16.4 | |
| SafeLoRAModel=Qwen-2.5-7B-Instruct, Utility Context=PubMedQA2025.03 | 17.1 | 16.4 | |
| RESTA-InstructModel=Qwen-2-7B-Instruct, Utility Context=GSM8K2025.03 | 17.4 | 17.1 | |
| RESTA-InstructModel=Qwen-2-7B-Instruct, Utility Context=PubMedQA2025.03 | 17.6 | 16.9 | |
| RESTA-InstructModel=Qwen-2.5-7B-Instruct, Utility Context=GSM8K2025.03 | 17.7 | 17.1 | |
| OriginalModel=Qwen-2-7B-Instruct, Utility Context=GSM8K2025.03 | 18.2 | 17.6 | |
| OriginalModel=Qwen-2-7B-Instruct, Utility Context=PubMedQA2025.03 | 18.2 | 17.6 | |
| RESTAModel=Qwen-2.5-7B-Instruct, Utility Context=GSM8K2025.03 | 18.3 | 17.6 | |
| RESTAModel=Qwen-2-7B-Instruct, Utility Context=PubMedQA2025.03 | 18.5 | 18 | |
| RESTAModel=Qwen-2-7B-Instruct, Utility Context=GSM8K2025.03 | 18.8 | 18.4 | |
| SafeLoRAModel=Qwen-2.5-7B-Instruct, Utility Context=GSM8K2025.03 | 19.4 | 18.6 | |
| SafeLoRAModel=Qwen-2-7B-Instruct, Utility Context=PubMedQA2025.03 | 19.5 | 19.1 | |
| Fine-tunedModel=Qwen-2.5-7B-Instruct, Utility Context=PubMedQA2025.03 | 20.2 | 19.4 | |
| Fine-tunedModel=Qwen-2.5-7B-Instruct, Utility Context=GSM8K2025.03 | 22.1 | 21.4 | |
| SafeLoRAModel=Qwen-2-7B-Instruct, Utility Context=GSM8K2025.03 | 22.3 | 21.7 | |
| Fine-tunedModel=Llama-3.1-8B-Instruct, Utility Context=PubMedQA2025.03 | 23.5 | 23.1 | |
| Fine-tunedModel=Qwen-2-7B-Instruct, Utility Context=GSM8K2025.03 | 25.3 | 24.8 | |
| Fine-tunedModel=Qwen-2-7B-Instruct, Utility Context=PubMedQA2025.03 | 26 | 25.7 | |
| Fine-tunedModel=Llama-2-7B-Chat, Utility Context=GSM8K2025.03 | 27.8 | 27.3 | |
| Fine-tunedModel=Llama-3.1-8B-Instruct, Utility Context=GSM8K2025.03 | 28.3 | 27.6 |