Multi-turn response selection on Ubuntu Dialogue Corpus V1 (test)
91.8R10@1Dial-MAE
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Dial-MAElearning rate=1e-4, batch size=1282023.06 | 91.8 | — | 96.4 | 99.3 | |
| Uni-Enc+BERT-FPProtocol=post-trained2021.06 | 91.6 | — | 96.5 | 99.4 | |
| BERT-FPModel Type=PrLMs-based2022.03 | 91.1 | — | 96.2 | 99.4 | |
| BERT-FPProtocol=post-trained2021.06 | 91.1 | — | 96.2 | 99.4 | |
| BERT-FP2023.06 | 91.1 | — | 96.2 | 99.4 | |
| BERT_TLModel Type=Our Model2022.03 | 91 | — | 96.2 | 99.3 | |
| DR-BERT2023.06 | 91 | — | 96.2 | 99.3 | |
| BERT-TL2023.06 | 91 | — | 96.2 | 99.3 | |
| BERT+CLtraining=fine-tuning BERT using contrastive learning2023.06 | 88.7 | — | 94.8 | 98.9 | |
| BERT-UMS+FGCProtocol=post-trained2021.06 | 88.6 | — | 94.8 | 99 | |
| Uni-EncoderProtocol=fine-tuned2021.06 | 88.6 | — | 94.6 | 98.9 | |
| BERT-SLArchitecture=PLM-based2020.09 | 88.4 | 97.5 | 94.6 | 99 | |
| BERT-SLModel Type=PrLMs-based2022.03 | 88.4 | — | 94.6 | 99 | |
| BERT-SLProtocol=fine-tuned2021.06 | 88.4 | — | 94.6 | 99 | |
| BERT-SL2023.06 | 88.4 | — | 94.6 | 99 | |
| BERT-SL w/o. URArchitecture=PLM-based, Ablation=Without Utterance Relevance2020.09 | 88.1 | 97.4 | 94.5 | 99 | |
| BERT-SL w/o. CDArchitecture=PLM-based, Ablation=Without Context Detection2020.09 | 88 | 97.3 | 94.5 | 98.9 | |
| BERT-SL w/o. NSPArchitecture=PLM-based, Ablation=Without Next Sentence Prediction2020.09 | 87.9 | 97.3 | 94.4 | 98.9 | |
| BERT-SL w/o. IDArchitecture=PLM-based, Ablation=Without Inconsistent Detection2020.09 | 87.7 | 97.2 | 94.2 | 98.9 | |
| UMS-BERT+Model Type=PrLMs-based2022.03 | 87.5 | — | 94.2 | 98.8 | |
| UMS-ELECTRA+Model Type=PrLMs-based2022.03 | 87.5 | — | 94.1 | 98.8 | |
| UMSBERT+Protocol=post-trained2021.06 | 87.5 | — | 94.2 | 98.8 | |
| UMS BERT+2023.06 | 87.5 | — | 94.2 | 98.8 | |
| BERT-VFTArchitecture=PLM-based, Implementation=Our implementation2020.09 | 86.7 | 96.9 | 93.9 | 98.7 | |
| BERT-VFTModel Type=PrLM-based2022.03 | 86.7 | — | 93.9 | 98.7 | |
| SA-BERT+HCLProtocol=fine-tuned2021.06 | 86.7 | — | 94 | 99.2 | |
| MDFNModel Type=PrLMs-based2022.03 | 86.6 | — | 93.2 | 98.4 | |
| MDFNProtocol=fine-tuned2021.06 | 86.6 | — | 93.2 | 98.4 | |
| BERT-VFT(DA)Data Augmentation=true2019.08 | 85.8 | — | 93.1 | 98.5 | |
| SA-BERTspeaker_aware=true2020.04 | 85.5 | 96.5 | 92.8 | 98.3 | |
| BERT-VFTFeature=Variable Fine-Tuning layers2019.08 | 85.5 | — | 92.8 | 98.5 | |
| SA-BERTArchitecture=PLM-based2020.09 | 85.5 | 96.5 | 92.8 | 98.3 | |
| BERT-VFTArchitecture=PLM-based, Implementation=Whang et al., 20202020.09 | 85.5 | — | 92.8 | 98.5 | |
| SA-BERTModel Type=PrLM-based2022.03 | 85.5 | — | 92.8 | 98.3 | |
| SA-BERTProtocol=fine-tuned2021.06 | 85.5 | — | 92.8 | 98.3 | |
| BERT-VFT2023.06 | 85.5 | — | 92.8 | 98.5 | |
| SA-BERT2023.06 | 85.5 | — | 92.8 | 98.3 | |
| BERT-DPTPost-training=Domain-specific corpus2019.08 | 85.1 | — | 92.4 | 98.4 | |
| ELECTRAModel Type=PrLMs-based2022.03 | 84.5 | — | 91.9 | 97.9 | |
| UMSBERTProtocol=fine-tuned2021.06 | 84.3 | — | 92 | 98.2 | |
| BERT+FGCProtocol=fine-tuned2021.06 | 82.9 | — | 91 | 98 | |
| RoBERTa-SS-DA2023.06 | 82.6 | — | 90.9 | 97.8 | |
| BERT_base2019.08 | 81.7 | — | 90.4 | 97.7 | |
| BERTArchitecture=PLM-based2020.09 | 81.7 | 95.4 | 90.4 | 97.7 | |
| BERTspeaker_aware=false2020.04 | 80.8 | 95 | 89.7 | 97.5 | |
| BERTModel Type=PrLM-based2022.03 | 80.8 | — | 89.7 | 97.5 | |
| BERTProtocol=fine-tuned2021.06 | 80.8 | — | 89.7 | 97.5 | |
| IMNEnsemble=true2019.01 | 80.7 | 95.1 | 90 | 97.8 | |
| MSN2020.04 | 80 | — | 89.9 | 97.8 | |
| MSN2019.08 | 80 | — | 89.9 | 97.8 | |
| MSNArchitecture=Non-PLM-based2020.09 | 80 | — | 89.9 | 97.8 | |
| MSNModel Type=non PrLM-based2022.03 | 80 | — | 89.9 | 97.8 | |
| MSN2023.06 | 80 | — | 89.9 | 97.8 | |
| IoI2020.04 | 79.6 | 94.7 | 89.4 | 97.4 | |
| IoI2019.08 | 79.6 | — | 89.4 | 97.4 | |
| ESIMArchitecture=Non-PLM-based2020.09 | 79.6 | 95 | 87.4 | 97.5 | |
| IoIArchitecture=Non-PLM-based2020.09 | 79.6 | 94.7 | 89.4 | 97.4 | |
| IoIModel Type=non PrLM-based2022.03 | 79.6 | — | 89.4 | 97.4 | |
| IOI2023.06 | 79.6 | — | 89.4 | 97.4 | |
| ESIM2023.06 | 79.6 | — | 89.4 | 97.5 | |
| IMNEnsemble=false2019.01 | 79.4 | 94.6 | 88.9 | 97.4 | |
| IMN2020.04 | 79.4 | 94.6 | 88.9 | 97.4 | |
| IMNArchitecture=Non-PLM-based2020.09 | 79.4 | 94.6 | 88.9 | 97.4 | |
| IMNModel Type=non PrLM-based2022.03 | 79.4 | — | 88.9 | 97.4 | |
| MRFN2020.04 | 78.6 | 94.5 | 88.6 | 97.6 | |
| MRFN2019.08 | 78.6 | — | 88.6 | 97.6 | |
| MRFNArchitecture=Non-PLM-based2020.09 | 78.6 | 94.5 | 88.6 | 97.6 | |
| MRFNModel Type=non PrLM-based2022.03 | 78.6 | — | 88.6 | 97.6 | |
| DAM2019.01 | 76.7 | 93.8 | 87.4 | 96.9 | |
| DAM2020.04 | 76.7 | 93.8 | 87.4 | 96.9 | |
| DAM2019.08 | 76.7 | — | 87.4 | 96.9 | |
| DAMArchitecture=Non-PLM-based2020.09 | 76.7 | 93.8 | 87.4 | 96.9 | |
| DAMModel Type=non PrLM-based2022.03 | 76.7 | — | 87.4 | 96.9 | |
| DAM2023.06 | 76.7 | — | 87.4 | 96.9 | |
| DUA2019.01 | 75.2 | — | 86.8 | 96.2 | |
| DUA2020.04 | 75.2 | — | 86.8 | 96.2 | |
| DUA2019.08 | 75.2 | — | 86.8 | 96.2 | |
| DUAArchitecture=Non-PLM-based2020.09 | 75.2 | — | 86.8 | 96.2 | |
| DUAModel Type=non PrLM-based2022.03 | 75.2 | — | 86.8 | 96.2 | |
| DUA2023.06 | 75.2 | — | 86.8 | 96.2 | |
| AK-DE-biGRU2019.08 | 74.7 | — | 86.8 | 97.2 | |
| SMN2019.01 | 72.6 | 92.6 | 84.7 | 96.1 | |
| SMN2020.04 | 72.6 | 92.6 | 84.7 | 96.1 | |
| SMN2019.08 | 72.6 | — | 84.7 | 96.1 | |
| SMNArchitecture=Non-PLM-based2020.09 | 72.6 | 92.6 | 84.7 | 96.1 | |
| SMNModel Type=non PrLM-based2022.03 | 72.6 | — | 84.7 | 96.1 | |
| SMN2023.06 | 72.6 | — | 84.7 | 96.1 | |
| HRDE-LTC2019.01 | 68.4 | 91.6 | 82.2 | 96 | |
| BiMPM2019.01 | 66.5 | 89.7 | 78.6 | 93.8 | |
| MultiView2019.08 | 66.2 | — | 80.1 | 95.1 | |
| Multi-ViewArchitecture=Non-PLM-based2020.09 | 66.2 | 90.8 | 80.1 | 95.1 | |
| DualEncoder_lstmEncoder=LSTM2019.08 | 63.8 | — | 78.4 | 94.9 | |
| DualLSTMArchitecture=Non-PLM-based2020.09 | 63.8 | 90.1 | 78.4 | 94.9 | |
| LSTMModel Type=non PrLM-based2022.03 | 63.8 | — | 78.4 | 94.9 | |
| LSTM2023.06 | 63.8 | — | 78.4 | 94.9 | |
| CompAgg2019.01 | 63.1 | 88.4 | 75.3 | 92.7 | |
| DualEncoder_bilstmEncoder=BiLSTM2019.08 | 63 | — | 78 | 94.4 | |
| DualEncoder_cnnEncoder=CNN2019.08 | 54.9 | — | 68.4 | 89.6 | |
| CNN2023.06 | 54.9 | — | 68.4 | 89.6 | |
| TF-IDF2023.06 | 41 | — | 54.5 | 70.8 |