Dialogue Generation on PersonaChat (test)
2.31Persona ConsistencyLMEDR
Evaluation results
| Method | Links | |||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LMEDRbackbone=BART-large2023.01 | 2.31 | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.57 | — | 2.94 | — | — | — | — | — | — | |
| P2BOT2023.01 | 2.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.51 | — | 2.86 | — | — | — | — | — | — | |
| LIC2023.01 | 2.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.27 | — | 2.69 | — | — | — | — | — | — | |
| GPT2-D3Backbone=GPT2, Data Manipulation=D3, Training=Fine-tuned2022.04 | 0.548 | 15.69 | — | — | 4.184 | 1.429 | 0.1835 | 4.614 | 6.426 | 7.321 | 2.267 | 9.803 | 18.2 | 0.557 | 2.532 | 2.255 | — | — | — | — | — | — | — | |
| GPT2-FILTERBackbone=GPT2, Data Manipulation=Entropy Filter, Training=Fine-tuned2022.04 | 0.537 | 16.9 | — | — | 3.734 | 1.337 | 0.1788 | 4.57 | 6.352 | 7.263 | 2.148 | 9.031 | 16.52 | 0.571 | 2.527 | 2.233 | — | — | — | — | — | — | — | |
| HumanType=Human Baseline2022.04 | 0.531 | — | — | — | — | — | — | 5.68 | 8.913 | 10.27 | 5.259 | 34.9 | 66.37 | 0.472 | 2.625 | 2.451 | — | — | — | — | — | — | — | |
| TRANS-D3Backbone=Transformer, Data Manipulation=D3, Training=From scratch2022.04 | 0.513 | 37.3 | — | — | 3.358 | 1.206 | 0.1574 | 4.223 | 6.165 | 7.298 | 1.826 | 7.923 | 14.42 | 0.485 | 2.397 | 2.172 | — | — | — | — | — | — | — | |
| GPT2Backbone=GPT2, Data Manipulation=None, Training=Fine-tuned2022.04 | 0.508 | 17.63 | — | — | 3.761 | 1.278 | 0.1693 | 4.485 | 6.187 | 7.029 | 2.011 | 8.26 | 15.03 | 0.518 | 2.508 | 2.243 | — | — | — | — | — | — | — | |
| TRANS-FILTERBackbone=Transformer, Data Manipulation=Entropy Filter, Training=From scratch2022.04 | 0.492 | 38.99 | — | — | 2.946 | 1.101 | 0.1563 | 4.283 | 6.033 | 7.088 | 1.796 | 7.696 | 14.06 | 0.446 | 2.318 | 2.088 | — | — | — | — | — | — | — | |
| GPT2-BTBackbone=GPT2, Data Manipulation=Back Translation, Training=Fine-tuned2022.04 | 0.454 | 16.96 | — | — | 3.943 | 1.348 | 0.1663 | 4.547 | 6.248 | 7.089 | 1.947 | 8.113 | 14.94 | 0.509 | 2.488 | 2.259 | — | — | — | — | — | — | — | |
| GPT2-CVAEBackbone=GPT2, Data Manipulation=CVAE, Training=Fine-tuned2022.04 | 0.426 | 17.16 | — | — | 3.339 | 1.36 | 0.1592 | 4.245 | 5.691 | 6.49 | 1.748 | 6.799 | 12.19 | 0.484 | 2.358 | 2.15 | — | — | — | — | — | — | — | |
| Transfer Transfo+PEConditioning=Self Persona2021.11 | 0.35 | 17.41 | 21.1 | 82 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TRANS-BTBackbone=Transformer, Data Manipulation=Back Translation, Training=From scratch2022.04 | 0.35 | 37.92 | — | — | 3.315 | 1.082 | 0.1527 | 4.274 | 5.905 | 6.752 | 1.76 | 7.108 | 13.39 | 0.289 | 2.337 | 2.142 | — | — | — | — | — | — | — | |
| Persona GeneratorConditioning=Self Persona2021.11 | 0.34 | 17.31 | 21.1 | 81.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TRANS-CVAEBackbone=Transformer, Data Manipulation=CVAE, Training=From scratch2022.04 | 0.335 | 37.61 | — | — | 3.312 | 1.191 | 0.1533 | 3.974 | 5.451 | 6.267 | 1.459 | 5.795 | 11.16 | 0.26 | 2.333 | 2.111 | — | — | — | — | — | — | — | |
| Transfer TransfoConditioning=Self Persona2021.11 | 0.32 | 17.78 | 21.5 | 80.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TRANSBackbone=Transformer, Data Manipulation=None, Training=From scratch2022.04 | 0.304 | 38.28 | — | — | 3.14 | 1.148 | 0.1486 | 4.046 | 5.484 | 6.262 | 1.609 | 6.298 | 11.71 | 0.235 | 2.303 | 2.038 | — | — | — | — | — | — | — | |
| Transfer Transfo+PEConditioning=Their Persona2021.11 | 0.28 | 18.15 | 21.8 | 77.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Persona GeneratorConditioning=Their Persona2021.11 | 0.28 | 18.11 | 21.9 | 76.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Transfer TransfoConditioning=Their Persona2021.11 | 0.25 | 18.31 | 22.3 | 76.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Persona ApproximatorConditioning=Self Persona2021.11 | 0.22 | 18.33 | 19.8 | 73.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Persona ApproximatorConditioning=Their Persona2021.11 | 0.16 | 18.59 | 20.4 | 71.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPMNConditioning=Self Persona2021.11 | 0.15 | 36.11 | 13.5 | 54.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeepCopyConditioning=Self Persona2021.11 | 0.11 | 36.35 | 12.2 | 52.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPMNConditioning=Their Persona2021.11 | 0.1 | 36.45 | 14.8 | 51.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeepCopyConditioning=Their Persona2021.11 | 0.07 | 36.77 | 13.7 | 49.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Transfer Transfo w/o personaConditioning=None2021.11 | 0.04 | 19.87 | 18.4 | 67.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BARTTraining=fine-tuning, Beam search width=52023.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 9.95 | 3.38 | 8.69 | 6.55 | — | — | |
| BART + DialoGPSTraining=fine-tuning, Beam search width=52023.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 10.97 | 4.08 | 9.26 | 6.7 | — | — | |
| DialoFlowTraining=fine-tuning, Beam search width=52023.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 10.46 | 3.03 | 9.32 | 6.89 | — | — | |
| GCNBackbone=LLaMA-3.2-3B, Decoding=greedy2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 10.11 | 3.01 | 14.54 | — | 21.43 | 16.46 | |
| GCNBackbone=Qwen2.5-3B, Decoding=greedy2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 9.78 | 3.12 | 14.47 | — | 20.9 | 16.59 | |
| GCNBackbone=GPT-2, Decoding=greedy2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 6.37 | 1.5 | 13.06 | — | 15.17 | 13.52 | |
| HyPEBackbone=LLaMA-3.2-3B, Decoding=greedy2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 11.12 | 3.84 | 15.92 | — | 22.64 | 17.5 | |
| HyPEBackbone=Qwen2.5-3B, Decoding=greedy2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 10.5 | 3.58 | 15.9 | — | 22.18 | 17.81 | |
| HyPEBackbone=GPT-2, Decoding=greedy2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 7.31 | 1.59 | 13.22 | — | 17.94 | 14.46 | |
| HyPE-baseBackbone=LLaMA-3.2-3B, Decoding=greedy2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 10.07 | 3.17 | 14.53 | — | 21.32 | 16.43 | |
| HyPE-baseBackbone=Qwen2.5-3B, Decoding=greedy2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 9.86 | 3.29 | 15.19 | — | 21.25 | 17.13 | |
| HyPE-baseBackbone=GPT-2, Decoding=greedy2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 7.28 | 1.68 | 13.47 | — | 17.05 | 14.16 | |
| MeanPoolBackbone=LLaMA-3.2-3B, Decoding=greedy2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 11.01 | 2.02 | 14.82 | — | 22.31 | 17.2 | |
| MeanPoolBackbone=Qwen2.5-3B, Decoding=greedy2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 9.54 | 3.15 | 14.82 | — | 20.39 | 17.09 | |
| MeanPoolBackbone=GPT-2, Decoding=greedy2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 7.31 | 1.54 | 12.47 | — | 17.59 | 14.25 | |
| ORIGBackbone=LLaMA-3.2-3B, Decoding=greedy2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 10.2 | 2.94 | 15.09 | — | 21.51 | 17.11 | |
| ORIGBackbone=Qwen2.5-3B, Decoding=greedy2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 9.76 | 2.74 | 14.53 | — | 21.11 | 16.39 | |
| ORIGBackbone=GPT-2, Decoding=greedy2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 5.59 | 1.81 | 8.67 | — | 12.68 | 14.59 | |
| Text BaselineBackbone=LLaMA-3.2-3B, Decoding=greedy2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 10.44 | 3.76 | 15.29 | — | 21.48 | 16.98 | |
| Text BaselineBackbone=Qwen2.5-3B, Decoding=greedy2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 10.24 | 2.75 | 15.11 | — | 21.64 | 17.1 | |
| Text BaselineBackbone=GPT-2, Decoding=greedy2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 5.06 | 1.05 | 8.85 | — | 12.63 | 12.11 |