Response Selection on Ubuntu (test)
0.886Recall@1 (Top 10)BERT-UMS+FGC
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| BERT-UMS+FGCModel category=PLM-based, Use FGC=true2021.11 | 0.886 | — | 0.948 | 0.99 | |
| BERT-DPT+FGCModel category=PLM-based, Use FGC=true2021.11 | 0.881 | — | 0.945 | 0.99 | |
| UMSBERT+Domain-specific post-training=true2020.09 | 0.875 | — | 0.942 | 0.988 | |
| UMSELECTRA+Domain-specific post-training=true2020.09 | 0.875 | — | 0.941 | 0.988 | |
| BERT-UMSModel category=PLM-based2021.11 | 0.875 | — | 0.942 | 0.988 | |
| BERT+Domain-specific post-training=true2020.09 | 0.862 | — | 0.935 | 0.987 | |
| BERT-DPTModel category=PLM-based2021.11 | 0.862 | — | 0.935 | 0.987 | |
| ELECTRA+Domain-specific post-training=true2020.09 | 0.861 | — | 0.932 | 0.985 | |
| SA-BERTDomain-specific post-training=false2020.09 | 0.855 | — | 0.928 | 0.983 | |
| UMSELECTRADomain-specific post-training=false2020.09 | 0.854 | — | 0.929 | 0.984 | |
| UMSBERTDomain-specific post-training=false2020.09 | 0.843 | — | 0.92 | 0.982 | |
| ELECTRA+FGCModel category=PLM-based, Use FGC=true2021.11 | 0.832 | — | 0.912 | 0.98 | |
| BERT+FGCModel category=PLM-based, Use FGC=true2021.11 | 0.829 | — | 0.91 | 0.98 | |
| ELECTRADomain-specific post-training=false2020.09 | 0.826 | — | 0.908 | 0.978 | |
| ELECTRAModel category=PLM-based2021.11 | 0.826 | — | 0.908 | 0.978 | |
| BERT (ours)Domain-specific post-training=false2020.09 | 0.82 | — | 0.906 | 0.978 | |
| BERTModel category=PLM-based2021.11 | 0.82 | — | 0.906 | 0.978 | |
| BERT-SS-DADomain-specific post-training=false2020.09 | 0.813 | — | 0.901 | 0.977 | |
| G-MSNtraining protocol=grayscale enhanced training2020.04 | 0.812 | 0.958 | 0.911 | 0.987 | |
| BERT (Gu et al. 2020)Domain-specific post-training=false2020.09 | 0.808 | — | 0.897 | 0.975 | |
| G-IOItraining protocol=grayscale enhanced training2020.04 | 0.805 | 0.951 | 0.902 | 0.981 | |
| MSN2020.04 | 0.8 | — | 0.899 | 0.978 | |
| MSNDomain-specific post-training=false2020.09 | 0.8 | — | 0.899 | 0.978 | |
| MSNModel category=non-PLM-based2021.11 | 0.8 | — | 0.899 | 0.978 | |
| BERT-Small+FGCModel category=PLM-based, Use FGC=true2021.11 | 0.8 | — | 0.89 | 0.974 | |
| IOI2020.04 | 0.796 | 0.947 | 0.894 | 0.974 | |
| IoIDomain-specific post-training=false2020.09 | 0.796 | — | 0.894 | 0.974 | |
| IoIModel category=non-PLM-based2021.11 | 0.796 | — | 0.894 | 0.974 | |
| IMNModel category=non-PLM-based2021.11 | 0.794 | — | 0.889 | 0.974 | |
| BERT-SmallModel category=PLM-based2021.11 | 0.792 | — | 0.888 | 0.972 | |
| G-DAMtraining protocol=grayscale enhanced training2020.04 | 0.789 | 0.946 | 0.891 | 0.986 | |
| MRFNModel category=non-PLM-based2021.11 | 0.786 | — | 0.886 | 0.976 | |
| DAM2020.04 | 0.767 | 0.938 | 0.874 | 0.969 | |
| DAMDomain-specific post-training=false2020.09 | 0.767 | — | 0.874 | 0.969 | |
| DAMModel category=non-PLM-based2021.11 | 0.767 | — | 0.874 | 0.961 | |
| G-SMNtraining protocol=grayscale enhanced training2020.04 | 0.765 | 0.938 | 0.873 | 0.969 | |
| DUA2020.04 | 0.752 | 0.926 | 0.868 | 0.962 | |
| DUADomain-specific post-training=false2020.09 | 0.752 | — | 0.868 | 0.962 | |
| DUAModel category=non-PLM-based2021.11 | 0.752 | — | 0.868 | 0.961 | |
| SMN2020.04 | 0.726 | 0.926 | 0.847 | 0.961 | |
| SMNDomain-specific post-training=false2020.09 | 0.726 | — | 0.847 | 0.961 | |
| SMNModel category=non-PLM-based2021.11 | 0.726 | — | 0.847 | 0.961 | |
| Multi-View2020.04 | 0.662 | 0.908 | 0.801 | 0.951 | |
| Multi-ViewDomain-specific post-training=false2020.09 | 0.662 | — | 0.801 | 0.951 | |
| Multi-ViewModel category=non-PLM-based2021.11 | 0.662 | — | 0.801 | 0.951 | |
| MV-LSTM2020.04 | 0.653 | 0.906 | 0.804 | 0.946 | |
| Match-LSTM2020.04 | 0.653 | 0.904 | 0.799 | 0.944 | |
| MV-LSTMDomain-specific post-training=false2020.09 | 0.653 | — | 0.804 | 0.946 | |
| Match-LSTMDomain-specific post-training=false2020.09 | 0.653 | — | 0.799 | 0.944 | |
| LSTM2020.04 | 0.638 | 0.901 | 0.784 | 0.949 | |
| LSTMDomain-specific post-training=false2020.09 | 0.638 | — | 0.784 | 0.949 | |
| BiLSTM2020.04 | 0.63 | 0.895 | 0.78 | 0.944 | |
| BiLSTMDomain-specific post-training=false2020.09 | 0.63 | — | 0.78 | 0.944 | |
| DL2R2020.04 | 0.626 | 0.899 | 0.783 | 0.944 | |
| DL2RDomain-specific post-training=false2020.09 | 0.626 | — | 0.783 | 0.944 | |
| CNN2020.04 | 0.549 | 0.848 | 0.684 | 0.896 | |
| CNNDomain-specific post-training=false2020.09 | 0.549 | — | 0.684 | 0.896 | |
| RNN2020.04 | 0.403 | 0.768 | 0.547 | 0.819 |