Open-domain Dialogue Evaluation on Free Run 2 1.0 (secondary data collection run)
0.455Overall Quality ScoreBi-Encoder Transformer
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Bi-Encoder Transformerpersona=false, n (total ratings)=6232022.03 | 0.455 | 0.635 | 0.629 | 0.728 | 0.924 | 0.922 | -0.443 | -0.212 | |
| Bi-Encoder Transformerpersona=true, n (total ratings)=5392022.03 | 0.423 | 0.747 | 0.763 | 0.555 | 0.728 | 0.474 | -0.348 | 0.04 | |
| Poly-Encoder Transformerpersona=false, n (total ratings)=5532022.03 | 0.344 | 0.464 | 0.407 | 0.554 | 0.763 | 0.822 | -0.338 | -0.266 | |
| Poly-Encoder Transformerpersona=true, n (total ratings)=6302022.03 | 0.26 | 0.464 | 0.372 | 0.56 | 0.581 | 0.496 | -0.412 | -0.238 | |
| Key-Value Memory Networkpersona=false, n (total ratings)=5392022.03 | 0.245 | 0.576 | 0.492 | 0.229 | 0.585 | 0.043 | -0.545 | 0.337 | |
| Key-Value Memory Networkpersona=true, n (total ratings)=6092022.03 | 0.154 | 0.453 | 0.29 | 0.027 | 0.544 | -0.2 | -0.515 | 0.382 | |
| Sequence to Sequencepersona=false, n (total ratings)=5952022.03 | 0.002 | 0.009 | -0.064 | 0.389 | 0.282 | 0.656 | -0.72 | -0.541 | |
| Language Modelpersona=false, n (total ratings)=5672022.03 | -0.202 | -0.063 | -0.044 | -0.075 | 0.3 | -0.346 | -0.646 | -0.539 | |
| Language Modelpersona=true, n (total ratings)=5112022.03 | -0.218 | -0.152 | -0.143 | 0.043 | 0.426 | -0.352 | -0.702 | -0.646 | |
| Sequence to Sequencepersona=true, n (total ratings)=6792022.03 | -0.258 | -0.285 | -0.304 | 0.033 | 0.209 | -0.226 | -0.55 | -0.683 |