Rationale Generation on SHINES
89RelevanceLlama-3.1-8B-Instruct
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Llama-3.1-8B-InstructProtocol=Multitask Fine-tuning + Prompting, CESM-100=true2025.06 | 89 | 89 | 88 | 88 | |
| Mental-Alpaca-7BProtocol=Multitask Fine-tuning + Prompting, CESM-100=true2025.06 | 87 | 87 | 87 | 86 | |
| Llama-3.1-8B-InstructProtocol=Multitask Fine-tuning + Prompting, CESM-100=false2025.06 | 86 | 85 | 85 | 85 | |
| Llama-3.1-8B-InstructProtocol=Fine-tuning + Prompting, CESM-100=true2025.06 | 86 | 86 | 85 | 85 | |
| Llama-3.1-8B-InstructProtocol=Fine-tuning + Prompting2025.06 | 85 | 85 | 84 | 84 | |
| MentaLLaMA-chat-7BProtocol=Multitask Fine-tuning + Prompting, CESM-100=true2025.06 | 85 | 85 | 85 | 85 | |
| Mental-Alpaca-7BProtocol=Multitask Fine-tuning + Prompting, CESM-100=false2025.06 | 84 | 84 | 84 | 83 | |
| Mental-Alpaca-7BProtocol=Fine-tuning + Prompting, CESM-100=true2025.06 | 84 | 85 | 84 | 84 | |
| Mental-Alpaca-7BProtocol=Fine-tuning + Prompting2025.06 | 83 | 83 | 83 | 83 | |
| MentaLLaMA-chat-7BProtocol=Multitask Fine-tuning + Prompting, CESM-100=false2025.06 | 83 | 83 | 82 | 82 | |
| MentaLLaMA-chat-7BProtocol=Fine-tuning + Prompting, CESM-100=true2025.06 | 83 | 83 | 82 | 82 | |
| MentaLLaMA-chat-7BProtocol=Fine-tuning + Prompting2025.06 | 81 | 82 | 80 | 81 | |
| Llama-3.1-8B-InstructProtocol=Few-shot Prompting2025.06 | 80 | 81 | 79 | 80 | |
| Mental-Alpaca-7BProtocol=Few-shot Prompting2025.06 | 78 | 79 | 78 | 79 | |
| MentaLLaMA-chat-7BProtocol=Few-shot Prompting2025.06 | 77 | 76 | 77 | 78 | |
| Llama-3.1-8B-InstructProtocol=Zero-shot Prompting2025.06 | 75 | 76 | 74 | 75 | |
| Mental-Alpaca-7BProtocol=Zero-shot Prompting2025.06 | 73 | 75 | 74 | 74 | |
| MentaLLaMA-chat-7BProtocol=Zero-shot Prompting2025.06 | 71 | 72 | 71 | 72 |