Multi-turn Response Selection on Ubuntu Corpus
86.9Recall@1 (R10)BERT + SPIDER
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| BERT + SPIDERTraining Mode=Domain Adaptive Post-training2021.05 | 86.9 | — | 93.8 | 98.7 | — | — | — | — | |
| BERTTraining Mode=Domain Adaptive Post-training2021.05 | 85.7 | — | 93 | 98.5 | — | — | — | — | |
| SA-BERTTraining Mode=Baseline2021.05 | 85.5 | — | 92.8 | 98.3 | — | — | — | — | |
| BERT + SPIDERTraining Mode=Multi-task Fine-tuning2021.05 | 83.1 | — | 91.3 | 98 | — | — | — | — | |
| TripleNet (Ensemble)Attention Mechanism=Yes, Model Ensemble=6 models2019.09 | 82.1 | 95.6 | 90.9 | 98 | — | — | — | — | |
| BERTTraining Mode=Multi-task Fine-tuning2021.05 | 81.7 | — | 90.4 | 97.7 | — | — | — | — | |
| TripleNet (ELMo)Attention Mechanism=Yes, Pre-training=ELMo2019.09 | 80.5 | 95.1 | 89.7 | 97.6 | — | — | — | — | |
| MSNTraining Mode=Baseline2021.05 | 80 | — | 89.9 | 97.8 | — | — | — | — | |
| IoITraining Mode=Baseline2021.05 | 79.6 | — | 89.4 | 97.4 | — | — | — | — | |
| TripleNetAttention Mechanism=Yes2019.09 | 79 | 94.3 | 88.5 | 97 | — | — | — | — | |
| MRFNTraining Mode=Baseline2021.05 | 78.6 | — | 88.6 | 97.6 | — | — | — | — | |
| DAMsource=Previous Research2019.08 | 76.7 | 93.8 | — | — | — | — | — | — | |
| DAMAttention Mechanism=Yes2019.09 | 76.7 | 93.8 | 87.4 | 96.9 | — | — | — | — | |
| DAMTraining Mode=Baseline2021.05 | 76.7 | — | 87.4 | 96.9 | — | — | — | — | |
| Multi-Granularity (5)backbone=Deep Attention Matching, number_of_models=52019.08 | 75.3 | 93.45 | — | — | 84.26 | — | — | — | |
| DUAType=Multi-turn matching2018.06 | 75.2 | — | 86.8 | 96.2 | — | — | — | — | |
| DUAAttention Mechanism=Yes2019.09 | 75.2 | — | 86.8 | 96.2 | — | — | — | — | |
| DUATraining Mode=Baseline2021.05 | 75.2 | — | 86.8 | 96.2 | — | — | — | — | |
| Ensemble (5)backbone=Deep Attention Matching, number_of_models=52019.08 | 74.95 | 93.27 | — | — | 84.03 | — | — | — | |
| DAMbackbone=Deep Attention Matching, re-trained=true2019.08 | 74.54 | 93.08 | — | — | 83.74 | — | — | — | |
| SMNdynamicWord embedding=word2vec, Embedding size=200, GRU hidden units=200, Convolution window size=(3,3), Feature maps=82016.12 | 72.6 | 92.6 | 84.7 | 96.1 | — | — | — | — | |
| SMNType=Multi-turn matching2018.06 | 72.6 | — | 84.7 | 96.1 | — | — | — | — | |
| SMNsource=Previous Research2019.08 | 72.6 | 92.6 | — | — | — | — | — | — | |
| SMNAttention Mechanism=None2019.09 | 72.6 | 92.6 | 84.7 | 96.1 | — | — | — | — | |
| SMNTraining Mode=Baseline2021.05 | 72.6 | — | 84.7 | 96.1 | — | — | — | — | |
| SMNstaticWord embedding=word2vec, Embedding size=200, GRU hidden units=200, Convolution window size=(3,3), Feature maps=82016.12 | 72.5 | 92.7 | 83.8 | 96.2 | — | — | — | — | |
| SMNlastWord embedding=word2vec, Embedding size=200, GRU hidden units=200, Convolution window size=(3,3), Feature maps=82016.12 | 72.3 | 92.3 | 84.2 | 95.6 | — | — | — | — | |
| Multi-Granularity (5)backbone=Dual Encoder, number_of_models=52019.08 | 68.7 | 91.9 | — | — | 80.1 | — | — | — | |
| RNN-CNNAttention Mechanism=Yes2019.09 | 67.2 | 91.1 | 80.9 | 95.6 | — | — | — | — | |
| Ensemble (5)backbone=Dual Encoder, number_of_models=52019.08 | 66.9 | 91.7 | — | — | 78.91 | — | — | — | |
| Multi-View2016.12 | 66.2 | 90.8 | 80.1 | 95.1 | — | — | — | — | |
| Multi-ViewType=Multi-turn matching2018.06 | 66.2 | — | 80.1 | 95.1 | — | — | — | — | |
| Multiviewsource=Previous Research2019.08 | 66.2 | 90.8 | — | — | — | — | — | — | |
| Multi-ViewAttention Mechanism=None2019.09 | 66.2 | 90.8 | 80.1 | 95.1 | — | — | — | — | |
| Multi-Channel2016.12 | 65.6 | 90.4 | 80.9 | 94.2 | — | — | — | — | |
| Multi-ChannelType=Single-turn matching2018.06 | 65.6 | — | 80.9 | 94.2 | — | — | — | — | |
| MV-LSTM2016.12 | 65.3 | 90.6 | 80.4 | 94.6 | — | — | — | — | |
| Match-LSTM2016.12 | 65.3 | 90.4 | 79.9 | 94.4 | — | — | — | — | |
| MV-LSTMType=Single-turn matching2018.06 | 65.3 | — | 80.4 | 94.6 | — | — | — | — | |
| Match-LSTMType=Single-turn matching2018.06 | 65.3 | — | 79.9 | 94.4 | — | — | — | — | |
| MV-LSTMsource=Previous Research2019.08 | 65.3 | 90.6 | — | — | — | — | — | — | |
| Match-LSTMsource=Previous Research2019.08 | 65.3 | 90.4 | — | — | — | — | — | — | |
| MV-LSTMAttention Mechanism=None2019.09 | 65.3 | 90.6 | 80.4 | 94.6 | — | — | — | — | |
| Match-LSTMAttention Mechanism=None2019.09 | 65.3 | 90.4 | 80.4 | 94.6 | — | — | — | — | |
| LSTM2016.12 | 63.8 | 90.1 | 78.4 | 94.9 | — | — | — | — | |
| LSTMType=Single-turn matching2018.06 | 63.8 | — | 78.4 | 94.9 | — | — | — | — | |
| Dual Encodersource=Previous Research2019.08 | 63.8 | 90.1 | — | — | — | — | — | — | |
| DualEncoderAttention Mechanism=None2019.09 | 63.8 | 90.1 | 78.4 | 94.9 | — | — | — | — | |
| Dual Encoderbackbone=Dual Encoder, experimental_context=Authors' implementation2019.08 | 63.6 | 90.9 | — | — | 76.84 | — | — | — | |
| Attentive-LSTM2016.12 | 63.3 | 90.3 | 78.9 | 94.3 | — | — | — | — | |
| Attentive-LSTMType=Single-turn matching2018.06 | 63.3 | — | 78.9 | 94.3 | — | — | — | — | |
| BiLSTM2016.12 | 63 | 89.5 | 78 | 94.4 | — | — | — | — | |
| BILSTMType=Single-turn matching2018.06 | 63 | — | 78 | 94.4 | — | — | — | — | |
| DL2R2016.12 | 62.6 | 89.9 | 78.3 | 94.4 | — | — | — | — | |
| DL2RType=Multi-turn matching2018.06 | 62.6 | — | 78.3 | 94.4 | — | — | — | — | |
| DL2Rsource=Previous Research2019.08 | 62.6 | 89.9 | — | — | — | — | — | — | |
| DL2RAttention Mechanism=None2019.09 | 62.6 | 89.9 | 78.3 | 94.4 | — | — | — | — | |
| CNN2016.12 | 54.9 | 84.8 | 68.4 | 89.6 | — | — | — | — | |
| CNNType=Single-turn matching2018.06 | 54.9 | — | 68.4 | 89.6 | — | — | — | — | |
| TF-IDF2016.12 | 41 | 65.9 | 54.5 | 70.8 | — | — | — | — | |
| TF-IDFType=Single-turn matching2018.06 | 41 | — | 54.5 | 70.8 | — | — | — | — | |
| RNN2016.12 | 40.3 | 76.8 | 54.7 | 81.9 | — | — | — | — | |
| RNNType=Single-turn matching2018.06 | 40.3 | — | 54.7 | 81.9 | — | — | — | — | |
| Multi-Channelexp2016.12 | 36.8 | 71.4 | 49.7 | 74.5 | — | — | — | — | |
| Multi-Channel_expType=Single-turn matching2018.06 | 36.8 | — | 49.7 | 74.5 | — | — | — | — | |
| Attentive-LSTM2019.01 | — | — | — | — | — | 63.3 | 78.9 | 94.3 | |
| BiLSTM2019.01 | — | — | — | — | — | 63 | 78 | 94.4 | |
| CNN2019.01 | — | — | — | — | — | 54.9 | 68.4 | 89.6 | |
| DAM2019.01 | — | — | — | — | — | 76.7 | 87.4 | 96.9 | |
| DL2R2019.01 | — | — | — | — | — | 62.6 | 78.3 | 94.4 | |
| DUA2019.01 | — | — | — | — | — | 75.2 | 86.8 | 96.2 | |
| ESIM2019.01 | — | — | — | — | — | 79.6 | 89.4 | 97.5 | |
| LSTM2019.01 | — | — | — | — | — | 63.8 | 78.4 | 94.9 | |
| Match-LSTM2019.01 | — | — | — | — | — | 65.3 | 79.9 | 94.4 | |
| Multi-Channel2019.01 | — | — | — | — | — | 65.6 | 80.9 | 94.2 | |
| Multi-View2019.01 | — | — | — | — | — | 66.2 | 80.1 | 95.1 | |
| MV-LSTM2019.01 | — | — | — | — | — | 65.3 | 80.4 | 94.6 | |
| RNN2019.01 | — | — | — | — | — | 40.3 | 54.7 | 81.9 | |
| SMN2019.01 | — | — | — | — | — | 72.6 | 84.7 | 96.1 | |
| TF-IDF2019.01 | — | — | — | — | — | 41 | 54.5 | 70.8 |