Human Evaluation (Win/Tie/Loss) on ESConv
70Win RateKEMI
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| KEMIComparison Baseline=MISC, Evaluation Aspect=Overall2023.05 | 70 | 12 | 18 | |
| PALBaseline=Blenderbot-Joint, Aspect=Coherence2022.12 | 68 | 6 | 26 | |
| KEMIComparison Baseline=BlenderBot-Joint, Evaluation Aspect=Overall2023.05 | 62 | 20 | 18 | |
| PALBaseline=MISC, Aspect=Comforting2022.12 | 62 | 14 | 24 | |
| PALBaseline=MISC, Aspect=Information2022.12 | 62 | 16 | 22 | |
| PRCCFComparisons=vs. D2RCU, Aspects=Comforting2026.04 | 60 | 7 | 33 | |
| PALBaseline=PAL (α = 0), Aspect=Identification2022.12 | 58 | 10 | 32 | |
| PRCCFComparisons=vs. D2RCU, Aspects=Identification2026.04 | 58 | 10 | 32 | |
| PRCCFComparisons=vs. D2RCU, Aspects=Overall2026.04 | 57 | 6 | 37 | |
| PALBaseline=MISC, Aspect=Coherence2022.12 | 54 | 12 | 34 | |
| PALBaseline=Blenderbot-Joint, Aspect=Suggestion2022.12 | 54 | 14 | 32 | |
| KEMIComparison Baseline=BlenderBot-Joint, Evaluation Aspect=Suggestion2023.05 | 52 | 22 | 26 | |
| KEMIComparison Baseline=MISC, Evaluation Aspect=Suggestion2023.05 | 52 | 16 | 28 | |
| PALBaseline=PAL (α = 0), Aspect=Information2022.12 | 52 | 4 | 44 | |
| PALBaseline=Blenderbot-Joint, Aspect=Overall2022.12 | 52 | 32 | 16 | |
| PRCCFComparisons=vs. DKPE, Aspects=Overall2026.04 | 51 | 12 | 37 | |
| KEMIComparison Baseline=BlenderBot-Joint, Evaluation Aspect=Identification2023.05 | 50 | 38 | 12 | |
| PALBaseline=Blenderbot-Joint, Aspect=Comforting2022.12 | 50 | 18 | 32 | |
| PRCCFComparisons=vs. DKPE, Aspects=Comforting2026.04 | 50 | 13 | 37 | |
| PRCCFComparisons=vs. D2RCU, Aspects=Suggestion2026.04 | 47 | 17 | 36 | |
| PRCCFComparisons=vs. DKPE, Aspects=Identification2026.04 | 47 | 17 | 36 | |
| KEMIComparison Baseline=BlenderBot-Joint, Evaluation Aspect=Comforting2023.05 | 46 | 40 | 14 | |
| KEMIComparison Baseline=MISC, Evaluation Aspect=Identification2023.05 | 46 | 30 | 24 | |
| PALBaseline=PAL (α = 0), Aspect=Coherence2022.12 | 46 | 6 | 48 | |
| PALBaseline=MISC, Aspect=Identification2022.12 | 46 | 12 | 42 | |
| PALBaseline=PAL (α = 0), Aspect=Suggestion2022.12 | 46 | 16 | 38 | |
| PRCCFComparisons=vs. DKPE, Aspects=Suggestion2026.04 | 46 | 26 | 28 | |
| KEMIComparison Baseline=MISC, Evaluation Aspect=Comforting2023.05 | 44 | 30 | 26 | |
| PALBaseline=PAL (α = 0), Aspect=Comforting2022.12 | 44 | 14 | 42 | |
| PALBaseline=Blenderbot-Joint, Aspect=Information2022.12 | 44 | 22 | 34 | |
| PALBaseline=MISC, Aspect=Overall2022.12 | 44 | 28 | 28 | |
| PALBaseline=Blenderbot-Joint, Aspect=Identification2022.12 | 42 | 14 | 44 | |
| PALBaseline=MISC, Aspect=Suggestion2022.12 | 42 | 16 | 42 | |
| PALBaseline=PAL (α = 0), Aspect=Overall2022.12 | 40 | 32 | 28 | |
| KEMIComparison Baseline=MISC, Evaluation Aspect=Fluency2023.05 | 37 | 47 | 16 | |
| KEMIComparison Baseline=BlenderBot-Joint, Evaluation Aspect=Fluency2023.05 | 26 | 51 | 23 |