Multi-turn Response Selection on E-commerce Dialogue Corpus (test)
93R@1 (Top 10 Set)Dial-MAE
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Dial-MAElearning rate=1e-4, batch size=1282023.06 | 93 | 97.7 | 99.7 | — | |
| BERT-TL2023.06 | 92.7 | 97.4 | 99.7 | — | |
| BERT-FP2023.06 | 87 | 95.6 | 99.3 | — | |
| BERT+CLtraining=fine-tuning BERT using contrastive learning2023.06 | 84.9 | 93.7 | 99.1 | — | |
| BERT-SLArchitecture=PLM-based2020.09 | 77.6 | 91.9 | 99.1 | — | |
| BERT-SL2023.06 | 77.6 | 91.9 | 99.1 | — | |
| UMS BERT+2023.06 | 76.4 | 90.5 | 98.6 | — | |
| BERT-SL w/o. URArchitecture=PLM-based, Ablation=Without Utterance Relevance2020.09 | 76.3 | 91.6 | 99.1 | — | |
| BERT-SL w/o. NSPArchitecture=PLM-based, Ablation=Without Next Sentence Prediction2020.09 | 76 | 91.4 | 98.8 | — | |
| BERT-SL w/o. IDArchitecture=PLM-based, Ablation=Without Inconsistent Detection2020.09 | 75.5 | 91.1 | 98.7 | — | |
| BERT-SL w/o. CDArchitecture=PLM-based, Ablation=Without Context Detection2020.09 | 74.2 | 89.7 | 98.6 | — | |
| BERT-VFTArchitecture=PLM-based, Implementation=Our implementation2020.09 | 71.7 | 88.4 | 98.6 | — | |
| SA-BERT2020.04 | 70.4 | 87.9 | 98.5 | — | |
| SA-BERTArchitecture=PLM-based2020.09 | 70.4 | 87.9 | 98.5 | — | |
| SA-BERT2023.06 | 70.4 | 87.9 | 98.5 | — | |
| IMNensemble=true2019.01 | 67.2 | 84.5 | 97 | — | |
| RoBERTa-SS-DA2023.06 | 62.7 | 83.5 | 98 | — | |
| IMNensemble=false2019.01 | 62.1 | 79.7 | 96.4 | — | |
| IMN2020.04 | 62.1 | 79.7 | 96.4 | — | |
| IMNArchitecture=Non-PLM-based2020.09 | 62.1 | 79.7 | 96.4 | — | |
| U2U-IMN2019.11 | 61.6 | 80.6 | 96.6 | 0.759 | |
| BERT2020.04 | 61 | 81.4 | 97.3 | — | |
| BERTArchitecture=PLM-based2020.09 | 61 | 81.4 | 97.3 | — | |
| MSN2020.04 | 60.6 | 77 | 93.7 | — | |
| MSNArchitecture=Non-PLM-based2020.09 | 60.6 | 77 | 93.7 | — | |
| MSN2023.06 | 60.6 | 77 | 93.7 | — | |
| ESIMArchitecture=Non-PLM-based2020.09 | 57 | 76.7 | 94.8 | — | |
| ESIM2023.06 | 57 | 76.7 | 94.8 | — | |
| IoI2020.04 | 56.3 | 76.8 | 95 | — | |
| IoIArchitecture=Non-PLM-based2020.09 | 56.3 | 76.8 | 95 | — | |
| IOI2023.06 | 56.3 | 76.8 | 95 | — | |
| DAMArchitecture=Non-PLM-based2020.09 | 52.6 | 72.7 | 93.3 | — | |
| DAM2023.06 | 52.6 | 72.7 | 93.3 | — | |
| DUA2018.06 | 50.1 | 70 | 92.1 | — | |
| DUA2019.11 | 50.1 | 70 | 92.1 | — | |
| DUAensemble=false2019.01 | 50.1 | 70 | 92.1 | — | |
| DUA2020.04 | 50.1 | 70 | 92.1 | — | |
| DUAArchitecture=Non-PLM-based2020.09 | 50.1 | 70 | 92.1 | — | |
| DUA2023.06 | 50.1 | 70 | 92.1 | — | |
| SMN2018.06 | 45.3 | 65.4 | 88.6 | — | |
| SMN2019.11 | 45.3 | 65.4 | 88.6 | — | |
| SMNensemble=false2019.01 | 45.3 | 65.4 | 88.6 | — | |
| SMN2020.04 | 45.3 | 65.4 | 88.6 | — | |
| SMNArchitecture=Non-PLM-based2020.09 | 45.3 | 65.4 | 88.6 | — | |
| SMN2023.06 | 45.3 | 65.4 | 88.6 | — | |
| Multi-Channel2018.06 | 42.2 | 60.9 | 87.1 | — | |
| Multi-View2018.06 | 42.1 | 60.1 | 86.1 | — | |
| Multi-View2019.11 | 42.1 | 60.1 | 86.1 | — | |
| Multi-ViewArchitecture=Non-PLM-based2020.09 | 42.1 | 60.1 | 86.1 | — | |
| MV-LSTM2018.06 | 41.2 | 59.1 | 85.7 | — | |
| MV-LSTM2019.11 | 41.2 | 59.1 | 85.7 | — | |
| Match-LSTM2018.06 | 41 | 59 | 85.8 | — | |
| Match-LSTM2019.11 | 41 | 59 | 85.8 | — | |
| Attentive-LSTM2018.06 | 40.1 | 58.1 | 84.9 | — | |
| DL2R2018.06 | 39.9 | 57.1 | 84.2 | — | |
| DL2R2019.11 | 39.9 | 57.1 | 84.2 | — | |
| LSTM2018.06 | 36.5 | 53.6 | 82.8 | — | |
| LSTM2019.11 | 36.5 | 53.6 | 82.8 | — | |
| DualLSTMArchitecture=Non-PLM-based2020.09 | 36.5 | 53.6 | 82.8 | — | |
| LSTM2023.06 | 36.5 | 53.6 | 82.8 | — | |
| BiLSTM2018.06 | 35.5 | 52.5 | 82.5 | — | |
| Multi-Channel (exp)2018.06 | 35.2 | 55.6 | 82.7 | — | |
| CNN2018.06 | 32.8 | 51.5 | 79.2 | — | |
| CNN2023.06 | 32.8 | 51.5 | 79.2 | — | |
| RNN2018.06 | 32.5 | 46.3 | 77.5 | — | |
| RNN2019.11 | 32.5 | 46.3 | 77.5 | — | |
| TF-IDF2018.06 | 15.9 | 25.6 | 47.7 | — | |
| TF-IDF2019.11 | 15.9 | 25.6 | 47.7 | — | |
| TF-IDF2023.06 | 15.9 | 25.6 | 47.7 | — | |
| RNN2023.06 | 11.8 | 22.3 | 58.9 | — |