Emotional Support Conversation on ESConv (test)
22.85BLEU-2AFlow
Evaluation results
| Method | Links | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AFlowCategory=Ours2026.02 | 22.85 | 8.95 | 24.85 | — | 63.5 | 9.25 | — | — | 5.25 | 21.5 | — | — | — | — | — | — | — | — | 23.55 | 65.1 | |
| Multi AgentCategory=Sim, Values & Empathy2026.02 | 20.5 | 9.45 | 22.5 | — | 60.5 | 7.85 | — | — | 4.45 | 19.5 | — | — | — | — | — | — | — | — | 21.5 | 62 | |
| EmoDxCategory=Strat & Know.2026.02 | 20.1 | 9.55 | 22.1 | — | 60.1 | 7.62 | — | — | 4.5 | 19.8 | — | — | — | — | — | — | — | — | 21.3 | 61.8 | |
| Value ReinfCategory=Sim, Values & Empathy2026.02 | 19.8 | 9.6 | 21.2 | — | 58.5 | 7.3 | — | — | 4.3 | 18.9 | — | — | — | — | — | — | — | — | 21 | 60.2 | |
| DYN-KnCategory=Strat & Know.2026.02 | 19.8 | 9.8 | 21.8 | — | 58.8 | 7.4 | — | — | 4.3 | 18.9 | — | — | — | — | — | — | — | — | 21 | 60.5 | |
| Social SimCategory=Sim, Values & Empathy2026.02 | 19.5 | 9.85 | 21.6 | — | 57.5 | 7.25 | — | — | 4.25 | 18.5 | — | — | — | — | — | — | — | — | 20.9 | 59.8 | |
| IntentESCCategory=Task & Interp2026.02 | 19.1 | 9.95 | 21.2 | — | 57.8 | 6.95 | — | — | 4.15 | 17.8 | — | — | — | — | — | — | — | — | 20.5 | 59.5 | |
| ESCoTCategory=Task & Interp2026.02 | 18.2 | 10.5 | 20.1 | — | 54.5 | 5.62 | — | — | 3.85 | 16.5 | — | — | — | — | — | — | — | — | 19.2 | 56.8 | |
| Claude 3.5Category=General LLMs2026.02 | 17.1 | — | 21.5 | — | 59.5 | 7.55 | — | — | 4.95 | 24.8 | — | — | — | — | — | — | — | — | 24.2 | 61.2 | |
| GPT-4oCategory=General LLMs2026.02 | 16.2 | — | 19.8 | — | 55.2 | 6.85 | — | — | 4.12 | 22.1 | — | — | — | — | — | — | — | — | 24.1 | 58.4 | |
| KEMICategory=Strat & Know.2026.02 | 16.1 | 14.8 | 18.2 | — | 49.8 | 3.43 | — | — | 3.45 | 14.1 | — | — | — | — | — | — | — | — | 16.2 | 51.2 | |
| MISCCategory=Strat & Know.2026.02 | 14.5 | 15.6 | 16.8 | — | 43.2 | 2.96 | — | — | 3.12 | 12.5 | — | — | — | — | — | — | — | — | 15.1 | 46.5 | |
| BlenderBot-JointCategory=Task & Interp2026.02 | 12.1 | 16.4 | 17.3 | — | 28.5 | 2.53 | — | — | 1.85 | 7.2 | — | — | — | — | — | — | — | — | 13.8 | 32.1 | |
| PRCCF2026.04 | 10.71 | — | 19.78 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MIMECategory=Sim, Values & Empathy2026.02 | 9.5 | — | 15.5 | — | — | 2.2 | — | — | 1.4 | 7.5 | — | — | — | — | — | — | — | — | 12.1 | — | |
| MoELCategory=Sim, Values & Empathy2026.02 | 8.8 | — | 14.8 | — | — | 1.95 | — | — | 1.3 | 6.8 | — | — | — | — | — | — | — | — | 11.5 | — | |
| KEMI2023.05 | 8.31 | 15.92 | 17.05 | — | 24.66 | 2.51 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Bart-JointBackbone=Bart2023.07 | 7.92 | 16.05 | — | — | — | — | 19.99 | 3.93 | 4.24 | 21.98 | 43.33 | 0.635 | 0.402 | 0.723 | 0.475 | 18.85 | — | — | — | — | |
| MISC2023.05 | 7.62 | 16.08 | 16.4 | — | 19.89 | 2.19 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GLHG2023.05 | 7.57 | — | 16.37 | — | 15.67 | 2.13 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GLHG2023.07 | 7.57 | 15.67 | — | — | — | — | 19.66 | 3.74 | 3.5 | 21.61 | — | — | — | — | — | — | — | — | — | — | |
| SUPPORTER2023.07 | 7.49 | 15.37 | — | — | — | — | 19.5 | 3.58 | 4.93 | 27.73 | 53.78 | 0.743 | 0.409 | 0.681 | 0.472 | 18.37 | — | — | — | — | |
| ChatGPT w/Example ExpansionPrompting strategy=Example Expansion2026.04 | 7.45 | — | 15.22 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MISC2023.07 | 7.31 | 16.16 | — | — | — | — | — | — | 4.41 | 19.71 | — | — | — | — | — | — | — | — | — | — | |
| SUPPORTERAblation=w/o ExpertPolicy2023.07 | 7.23 | 15.54 | — | — | — | — | 18.3 | 3.54 | 4.75 | 27.23 | 52.85 | 0.683 | 0.395 | 0.657 | 0.454 | 18.54 | — | — | — | — | |
| ChatGLM-6B w/P-TuningBackbone=ChatGLM-6B, Adaptation method=P-Tuning2026.04 | 7.22 | — | 16.15 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BlenderBot-JointBackbone=BlenderBot2023.07 | 7.17 | 14.78 | — | — | — | — | 17.97 | 3.31 | 4.56 | 24.65 | 49.71 | 0.611 | 0.398 | 0.71 | 0.459 | 17.69 | — | — | — | — | |
| SUPPORTERAblation=w/o EmoExperts2023.07 | 7.12 | 15.35 | — | — | — | — | 18.32 | 3.38 | 4.79 | 27.2 | 53.01 | 0.711 | 0.392 | 0.679 | 0.46 | 18.14 | — | — | — | — | |
| SUPPORTERAblation=w/o ESRewards2023.07 | 7.1 | 15.46 | — | — | — | — | 18.49 | 3.36 | 4.69 | 26.92 | 52.49 | 0.664 | 0.391 | 0.66 | 0.457 | 18.41 | — | — | — | — | |
| SUPPORTERTraining=w/o Warm-Start2023.07 | 6.86 | 15.01 | — | — | — | — | 17.98 | 3.18 | 4.55 | 26.06 | 51.62 | 0.673 | 0.403 | 0.638 | 0.453 | 18.26 | — | — | — | — | |
| SUPPORTERAblation=w/o DCRewards2023.07 | 6.8 | 15.43 | — | — | — | — | 17.28 | 3.25 | 4.8 | 27.45 | 53.04 | 0.707 | 0.401 | 0.652 | 0.448 | 17.12 | — | — | — | — | |
| MoEL2023.07 | 6.77 | 112.34 | — | — | — | — | 18.14 | 3.22 | 2.43 | 17.03 | 38.08 | 0.658 | 0.39 | 0.391 | 0.384 | 20.36 | — | — | — | — | |
| SUPPORTERAblation=w/o KwsExperts2023.07 | 6.74 | 15.54 | — | — | — | — | 17.76 | 3.19 | 4.69 | 26.16 | 50.92 | 0.728 | 0.394 | 0.636 | 0.443 | 17.72 | — | — | — | — | |
| SUPPORTERTraining=Warm-Start Only2023.07 | 6.74 | 15.03 | — | — | — | — | 17.42 | 3.21 | 4.67 | 26.24 | 51.82 | 0.629 | 0.402 | 0.644 | 0.444 | 17.35 | — | — | — | — | |
| MIME2023.07 | 6.58 | 68.49 | — | — | — | — | 15.89 | 3.27 | 2.02 | 10.51 | 22.6 | 0.598 | 0.37 | 0.45 | 0.412 | 19.44 | — | — | — | — | |
| SUPPORTERAblation=w/o Multi-Task2023.07 | 6.54 | 15.49 | — | — | — | — | 16.79 | 3.18 | 4.78 | 27.17 | 53.45 | 0.651 | 0.399 | 0.651 | 0.45 | 16.48 | — | — | — | — | |
| BlenderBot (Oracle)Backbone=BlenderBot, Variant=Oracle2021.06 | 6.31 | 16.03 | 17.9 | 51.65 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| straQ*-distillMethod variant=distill2026.04 | 6.25 | — | 15.44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GLHGloss=without L2 Loss2023.05 | 6.15 | — | 15.87 | — | — | 1.75 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Transformer2023.05 | 5.66 | 11.55 | 14.68 | — | — | 1.31 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DialoGPT (Oracle)Backbone=DialoGPT, Variant=Oracle2021.06 | 5.52 | 15.19 | 15.82 | 50.18 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BlenderBot-Joint2023.05 | 5.52 | 16.15 | 15.51 | — | 19.23 | 1.29 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BlenderBot (Vanilla)Backbone=BlenderBot, Variant=Vanilla2021.06 | 5.45 | 16.23 | 15.43 | 50.49 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BlenderBot (Joint)Backbone=BlenderBot, Variant=Joint2021.06 | 5.35 | — | 15.46 | 50.27 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DialoGPT (Vanilla)Backbone=DialoGPT, Variant=Vanilla2021.06 | 5.13 | 15.51 | 15.26 | 49.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MoEL2023.05 | 5.02 | 12.93 | 14.21 | — | — | 1.14 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DialoGPT (Joint)Backbone=DialoGPT, Variant=Joint2021.06 | 5 | — | 15.09 | 49.97 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MIME2023.05 | 4.82 | 13.27 | 14.83 | — | — | 1.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ChatGPT 1-shotPrompting strategy=1-shot2026.04 | 4.53 | — | 13.19 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FSM2026.04 | 3.25 | — | 15.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ask-an-ExpertReference Model=SFT2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.34 | 0.18 | — | — | |
| Ask-an-ExpertReference Model=DPO (GPT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.34 | 0.14 | — | — | |
| Ask-an-ExpertReference Model=DPO (SFT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.32 | 0.15 | — | — | |
| BlenderBot2023.05 | — | 15.45 | 15.43 | — | 16.23 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DPO (GPT)Category=Emotion Reinforced, h=3, gamma=1, Tdiff=0.5, Reference Model=SFT2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.52 | 0.55 | — | — | |
| DPO (GPT)Category=Emotion Reinforced, h=3, gamma=1, Tdiff=0.5, Reference Model=DPO (GPT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.49 | 0.49 | — | — | |
| DPO (GPT)Category=Emotion Reinforced, h=3, gamma=1, Tdiff=0.5, Reference Model=DPO (SFT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.49 | 0.51 | — | — | |
| DPO (GPT)Category=Emotion Reinforced, h=5, gamma=1, Tdiff=0.5, Reference Model=SFT2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.53 | 0.59 | — | — | |
| DPO (GPT)Category=Emotion Reinforced, h=5, gamma=1, Tdiff=0.5, Reference Model=DPO (GPT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.49 | 0.44 | — | — | |
| DPO (GPT)Category=Emotion Reinforced, h=5, gamma=1, Tdiff=0.5, Reference Model=DPO (SFT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.49 | 0.48 | — | — | |
| ES-VR SFTReference Model=DPO (GPT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.48 | 0.44 | — | — | |
| ES-VR SFTReference Model=DPO (SFT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.48 | 0.46 | — | — | |
| ESCOTReference Model=SFT2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.28 | 0.06 | — | — | |
| ESCOTReference Model=DPO (GPT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.25 | 0.04 | — | — | |
| ESCOTReference Model=DPO (SFT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.25 | 0.05 | — | — | |
| GPTReference Model=SFT2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.51 | 0.48 | — | — | |
| GPTReference Model=DPO (GPT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.49 | 0.42 | — | — | |
| GPTReference Model=DPO (SFT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.49 | 0.42 | — | — | |
| LlamaReference Model=SFT2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.5 | 0.51 | — | — | |
| LlamaReference Model=DPO (GPT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.48 | 0.44 | — | — | |
| LlamaReference Model=DPO (SFT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.46 | 0.45 | — | — | |
| Llama-ESConvReference Model=SFT2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.37 | 0.21 | — | — | |
| Llama-ESConvReference Model=DPO (GPT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.35 | 0.17 | — | — | |
| Llama-ESConvReference Model=DPO (SFT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.37 | 0.19 | — | — | |
| Llama-ExTESReference Model=SFT2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.51 | 0.54 | — | — | |
| Llama-ExTESReference Model=DPO (GPT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.5 | 0.52 | — | — | |
| Llama-ExTESReference Model=DPO (SFT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.48 | 0.51 | — | — | |
| Llama-Psych8kReference Model=SFT2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.5 | 0.64 | — | — | |
| Llama-Psych8kReference Model=DPO (GPT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.48 | 0.58 | — | — | |
| Llama-Psych8kReference Model=DPO (SFT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.49 | 0.62 | — | — | |
| Llama-RedditReference Model=SFT2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.3 | 0.1 | — | — | |
| Llama-RedditReference Model=DPO (GPT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.28 | 0.07 | — | — | |
| Llama-RedditReference Model=DPO (SFT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.29 | 0.09 | — | — | |
| PPDPPReference Model=SFT2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.47 | 0.37 | — | — | |
| PPDPPReference Model=DPO (GPT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.46 | 0.27 | — | — | |
| PPDPPReference Model=DPO (SFT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.44 | 0.31 | — | — | |
| SFTCategory=Emotion Reinforced, Reference Model=DPO (GPT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.5 | 0.43 | — | — | |
| SFTCategory=Emotion Reinforced, Reference Model=DPO (SFT)2025.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.49 | 0.46 | — | — |