CBT Conversation Generation on CBT conversation evaluation dataset
1.94Semantic CoherenceDeepseek V3 671B
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Deepseek V3 671BType=Zero-shot Large-scale General LLMs, Parameters=671B2026.02 | 1.94 | 2.78 | 2.75 | 1 | 0.99 | 0.91 | |
| Deepseek R1 671BType=Zero-shot Large-scale General LLMs, Parameters=671B2026.02 | 1.9 | 2.84 | 2.78 | 1 | 0.99 | 0.95 | |
| SIF 7BType=Ours, Parameters=7B2026.02 | 1.9 | 2.79 | 2.82 | 1 | 0.99 | 0.97 | |
| Llama 7BType=Fine-tuned Small-scale Open-source LLMs, Parameters=7B2026.02 | 1.83 | 2.5 | 2.59 | 1 | 0.44 | 0.48 | |
| CBT-BENCH 7BType=Zero-shot Psychological LLMs, Parameters=7B2026.02 | 1.82 | 2.24 | 2.32 | 1 | 0.98 | 0.8 | |
| SoulChat 7BType=Zero-shot Psychological LLMs, Parameters=7B2026.02 | 1.75 | 2.43 | 2.57 | 1 | 0.19 | 0.16 | |
| CBT-LLM 7BType=Zero-shot Psychological LLMs, Parameters=7B2026.02 | 1.74 | 2.52 | 2.76 | 1 | 0.54 | 0.14 | |
| Qwen2 7BType=Fine-tuned Small-scale Open-source LLMs, Parameters=7B2026.02 | 1.74 | 2.64 | 2.54 | 1 | 0.99 | 0.96 | |
| Chatglm 7BType=Fine-tuned Small-scale Open-source LLMs, Parameters=7B2026.02 | 1.74 | 2.7 | 2.76 | 1 | 0.62 | 0.76 | |
| Alpaca 7BType=Fine-tuned Small-scale Open-source LLMs, Parameters=7B2026.02 | 1.49 | 2.05 | 2.09 | 1 | 0.34 | 0.81 |