Question Answering on TeleQnA (Utility)
70.1UtilitySafeInstruct
Evaluation results
| Method | Links | |
|---|---|---|
| SafeInstructModel=Qwen-2.5-7B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 70.1 | |
| SafeMERGEModel=Qwen-2.5-7B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 70 | |
| Fine-tunedModel=Qwen-2.5-7B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 69.9 | |
| SafeLoRAModel=Qwen-2.5-7B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 68.4 | |
| Fine-tunedModel=Llama-3.1-8B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 67.8 | |
| SafeMERGEModel=Llama-3.1-8B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 67.1 | |
| SafeInstructModel=Llama-3.1-8B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 66.8 | |
| Fine-tunedModel=Qwen-2-7B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 65.6 | |
| SafeLoRAModel=Llama-3.1-8B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 65.3 | |
| SafeMERGEModel=Qwen-2-7B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 65.2 | |
| SafeInstructModel=Qwen-2-7B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 64.8 | |
| SafeLoRAModel=Qwen-2-7B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 64.1 | |
| RESTA-InstructModel=Llama-3.1-8B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 60.9 | |
| RESTA-InstructModel=Qwen-2-7B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 60.4 | |
| RESTA-InstructModel=Qwen-2.5-7B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 60.2 | |
| Fine-tunedModel=Llama-2-7B-Chat, Fine-tuning Dataset=TeleQnA2025.03 | 57.8 | |
| SafeMERGEModel=Llama-2-7B-Chat, Fine-tuning Dataset=TeleQnA2025.03 | 57.2 | |
| SafeLoRAModel=Llama-2-7B-Chat, Fine-tuning Dataset=TeleQnA2025.03 | 57 | |
| SafeInstructModel=Llama-2-7B-Chat, Fine-tuning Dataset=TeleQnA2025.03 | 56.3 | |
| RESTAModel=Llama-3.1-8B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 52.2 | |
| RESTAModel=Qwen-2.5-7B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 50.6 | |
| RESTAModel=Qwen-2-7B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 49.8 | |
| RESTA-InstructModel=Llama-2-7B-Chat, Fine-tuning Dataset=TeleQnA2025.03 | 49.6 | |
| OriginalModel=Qwen-2.5-7B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 48.7 | |
| OriginalModel=Qwen-2-7B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 45.8 | |
| OriginalModel=Llama-3.1-8B-Instruct, Fine-tuning Dataset=TeleQnA2025.03 | 42.3 | |
| RESTAModel=Llama-2-7B-Chat, Fine-tuning Dataset=TeleQnA2025.03 | 41.4 | |
| OriginalModel=Llama-2-7B-Chat, Fine-tuning Dataset=TeleQnA2025.03 | 35.8 |