CBT conversation generation on EmoLLM (test)
0.91Bert ScoreDeepseek V3 671B
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Deepseek V3 671BEvaluation Protocol=Zero-shot, Model Type=Large-scale General, Dialogue Mode=Multi-turn (Mul.)2026.02 | 0.91 | 0.41 | 52 | 0.7 | 0.39 | 0.47 | |
| CBT-BENCH 7BEvaluation Protocol=Zero-shot, Model Type=Psychological, Dialogue Mode=Multi-turn (Mul.)2026.02 | 0.91 | 0.4 | 27 | 0.65 | 0.19 | 0.31 | |
| Alpaca 7BEvaluation Protocol=Fine-tuned, Model Type=Small-scale Open-source, Dialogue Mode=Multi-turn (Mul.)2026.02 | 0.91 | 0.41 | 27 | 0.61 | 0.29 | 0.48 | |
| SIF 7BEvaluation Protocol=Ours, Dialogue Mode=Multi-turn (Mul.)2026.02 | 0.91 | 0.41 | 47 | 0.69 | 0.32 | 0.41 | |
| SoulChat 7BEvaluation Protocol=Zero-shot, Model Type=Psychological, Dialogue Mode=Multi-turn (Mul.)2026.02 | 0.9 | 0.41 | 25 | 0.65 | 0.25 | 0.39 | |
| Llama 7BEvaluation Protocol=Fine-tuned, Model Type=Small-scale Open-source, Dialogue Mode=Multi-turn (Mul.)2026.02 | 0.9 | 0.41 | 32 | 0.66 | 0.27 | 0.46 | |
| Qwen2 7BEvaluation Protocol=Fine-tuned, Model Type=Small-scale Open-source, Dialogue Mode=Multi-turn (Mul.)2026.02 | 0.9 | 0.41 | 46 | 0.67 | 0.3 | 0.41 | |
| Deepseek R1 671BEvaluation Protocol=Zero-shot, Model Type=Large-scale General, Dialogue Mode=Multi-turn (Mul.)2026.02 | 0.89 | 0.38 | 56 | 0.66 | 0.33 | 0.41 | |
| CBT-LLM 7BEvaluation Protocol=Zero-shot, Model Type=Psychological, Dialogue Mode=Multi-turn (Mul.)2026.02 | 0.89 | 0.41 | 12 | 0.58 | 0.13 | 0.26 | |
| Chatglm 7BEvaluation Protocol=Fine-tuned, Model Type=Small-scale Open-source, Dialogue Mode=Multi-turn (Mul.)2026.02 | 0.89 | 0.4 | 45 | 0.65 | 0.25 | 0.35 | |
| Deepseek R1 671BEvaluation Protocol=Zero-shot, Model Type=Large-scale General, Dialogue Mode=Single-utterance (Sig.)2026.02 | 0.69 | 0.45 | 57 | 0.06 | 0.35 | 0.26 | |
| Deepseek V3 671BEvaluation Protocol=Zero-shot, Model Type=Large-scale General, Dialogue Mode=Single-utterance (Sig.)2026.02 | 0.69 | 0.32 | 63 | 0.08 | 0.33 | 0.27 | |
| SIF 7BEvaluation Protocol=Ours, Dialogue Mode=Single-utterance (Sig.)2026.02 | 0.68 | 0.28 | 58 | 0.08 | 0.3 | 0.25 | |
| Qwen2 7BEvaluation Protocol=Fine-tuned, Model Type=Small-scale Open-source, Dialogue Mode=Single-utterance (Sig.)2026.02 | 0.67 | 0.21 | 56 | 0.08 | 0.27 | 0.23 | |
| CBT-BENCH 7BEvaluation Protocol=Zero-shot, Model Type=Psychological, Dialogue Mode=Single-utterance (Sig.)2026.02 | 0.65 | -0.28 | 59 | 0.08 | 0.16 | 0.18 | |
| Chatglm 7BEvaluation Protocol=Fine-tuned, Model Type=Small-scale Open-source, Dialogue Mode=Single-utterance (Sig.)2026.02 | 0.65 | -0.32 | 48 | 0.08 | 0.13 | 0.17 | |
| SoulChat 7BEvaluation Protocol=Zero-shot, Model Type=Psychological, Dialogue Mode=Single-utterance (Sig.)2026.02 | 0.64 | -0.1 | 55 | 0.06 | 0.15 | 0.14 | |
| CBT-LLM 7BEvaluation Protocol=Zero-shot, Model Type=Psychological, Dialogue Mode=Single-utterance (Sig.)2026.02 | 0.63 | -1.04 | 27 | 0.07 | 0.04 | 0.1 | |
| Llama 7BEvaluation Protocol=Fine-tuned, Model Type=Small-scale Open-source, Dialogue Mode=Single-utterance (Sig.)2026.02 | 0.62 | -0.67 | 36 | 0.06 | 0.08 | 0.13 | |
| Alpaca 7BEvaluation Protocol=Fine-tuned, Model Type=Small-scale Open-source, Dialogue Mode=Single-utterance (Sig.)2026.02 | 0.58 | -0.91 | 25 | 0.05 | 0.05 | 0.12 |