Response Selection on DSTC7 Track 1 (test)
91.1Recall@1 (Top 100)Cross-encoder
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Cross-encoderInitialization=This paper, Pre-training corpus=Reddit2019.04 | 91.1 | 71.7 | 92.4 | 79 | |
| Poly-encoderm=64, Mechanism=First-m, Initialization=This paper, Pre-training corpus=Reddit2019.04 | 91 | 70.9 | 91.3 | 78 | |
| Poly-encoderm=360, Mechanism=Learnt-m, Initialization=This paper, Pre-training corpus=Reddit2019.04 | 90.3 | 71.4 | 91.1 | 78.3 | |
| Cross-encoderPre-training Data=Reddit, Pre-training Source=Our pre-training2019.04 | 71.7 | — | — | 79 | |
| Poly-encoder 16Pre-training Data=Reddit, Pre-training Source=Our pre-training, m context vectors=162019.04 | 71.6 | — | — | 78.4 | |
| Poly-encoder 360Pre-training Data=Reddit, Pre-training Source=Our pre-training, m context vectors=3602019.04 | 71.4 | — | — | 78.3 | |
| Poly-encoder 64Pre-training Data=Reddit, Pre-training Source=Our pre-training, m context vectors=642019.04 | 71.2 | — | — | 78.2 | |
| ConveRTContext mode=multi context2019.11 | 71.2 | — | — | 78.8 | |
| Bi-encoderPre-training Data=Reddit, Pre-training Source=Our pre-training2019.04 | 70.9 | — | — | 78.1 | |
| Bi-encoder2019.11 | 70.9 | — | — | 78.1 | |
| Poly-encoder 360Pre-training Data=Toronto Books + Wikipedia, Pre-training Source=Devlin et al., 2019, m context vectors=3602019.04 | 68.9 | — | — | 76.2 | |
| Poly-encoder 16Pre-training Data=Toronto Books + Wikipedia, Pre-training Source=Devlin et al., 2019, m context vectors=162019.04 | 67.8 | — | — | 75.1 | |
| Cross-encoderPre-training Data=Toronto Books + Wikipedia, Pre-training Source=Devlin et al., 20192019.04 | 67.4 | — | — | 75.6 | |
| Poly-encoder 64Pre-training Data=Toronto Books + Wikipedia, Pre-training Source=Devlin et al., 2019, m context vectors=642019.04 | 67 | — | — | 74.7 | |
| Bi-encoderPre-training Data=Toronto Books + Wikipedia, Pre-training Source=Devlin et al., 20192019.04 | 66.8 | — | — | 74.6 | |
| Poly-encoder 64Pre-training Data=Toronto Books + Wikipedia, Pre-training Source=Our pre-training, m context vectors=642019.04 | 65.8 | — | — | 73.5 | |
| Poly-encoder 360Pre-training Data=Toronto Books + Wikipedia, Pre-training Source=Our pre-training, m context vectors=3602019.04 | 65.8 | — | — | 73.6 | |
| Poly-encoder 16Pre-training Data=Toronto Books + Wikipedia, Pre-training Source=Our pre-training, m context vectors=162019.04 | 65.3 | — | — | 73.2 | |
| Cross-encoderPre-training Data=Toronto Books + Wikipedia, Pre-training Source=Our pre-training2019.04 | 65.3 | — | — | 73.8 | |
| Chen & Wang2019.04 | 64.5 | — | 90.2 | 73.5 | |
| Chen & Wang2019.04 | 64.5 | — | — | 73.5 | |
| Bi-encoderPre-training Data=Toronto Books + Wikipedia, Pre-training Source=Our pre-training2019.04 | 64.5 | — | — | 72.6 | |
| Best DSTC7 System2019.11 | 64.5 | — | — | 73.5 | |
| Gu et al.2019.04 | 60.8 | — | — | 69.1 | |
| BERT2019.11 | 53 | — | — | 63.2 | |
| GPT2019.11 | 48.9 | — | — | 59.5 | |
| ConveRTContext mode=single context2019.11 | 38.2 | — | — | 49.2 |