Open-domain Dialogue Evaluation on Free run Mechanical Turk 1 (initial data collection run)
0.534Overall ScoreBi-Encoder Transformer
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Bi-Encoder Transformerpersona=false, n=7982022.03 | 0.534 | 0.564 | 0.602 | 0.711 | 0.863 | 0.964 | -0.038 | 0.069 | |
| Poly-Encoder Transformerpersona=false, n=7982022.03 | 0.419 | 0.474 | 0.481 | 0.614 | 0.875 | 0.994 | -0.431 | -0.075 | |
| Bi-Encoder Transformerpersona=true, n=7072022.03 | 0.318 | 0.399 | 0.372 | 0.443 | 0.821 | 0.404 | -0.33 | 0.116 | |
| Key-Value Memory Networkpersona=false, n=7912022.03 | 0.262 | 0.491 | 0.379 | 0.028 | 0.636 | -0.066 | -0.316 | 0.68 | |
| Key-Value Memory Networkpersona=true, n=7142022.03 | 0.189 | 0.409 | 0.373 | 0.159 | 0.672 | -0.114 | -0.521 | 0.349 | |
| Poly-Encoder Transformerpersona=true, n=7072022.03 | 0.173 | 0.23 | 0.197 | 0.369 | 0.673 | 0.32 | -0.395 | -0.187 | |
| Sequence to Sequencepersona=false, n=7072022.03 | -0.087 | -0.19 | -0.208 | 0.166 | 0.311 | 0.401 | -0.637 | -0.449 | |
| Sequence to Sequencepersona=true, n=7982022.03 | -0.201 | -0.308 | -0.234 | 0.092 | 0.312 | 0.025 | -0.625 | -0.669 | |
| LSTM-based Modelpersona=true, n=7632022.03 | -0.217 | -0.181 | -0.201 | -0.196 | 0.38 | -0.455 | -0.605 | -0.264 | |
| LSTM-based Modelpersona=false, n=7422022.03 | -0.243 | -0.165 | -0.16 | -0.142 | 0.329 | -0.407 | -0.745 | -0.411 |