Response Selection on E-commerce (test)
0.927Recall@1 (R10)BERT_TL
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BERT_TLModel Type=Our Model2022.03 | 0.927 | 0.974 | 0.997 | |
| BERT-FPModel Type=PrLMs-based2022.03 | 0.87 | 0.956 | 0.993 | |
| BERT-SLModel Type=PrLMs-based2022.03 | 0.776 | 0.919 | 0.991 | |
| UMSBERT+Domain-specific post-training=true2020.09 | 0.762 | 0.905 | 0.986 | |
| UMS-BERT+Model Type=PrLMs-based2022.03 | 0.762 | 0.905 | 0.986 | |
| BERT+Domain-specific post-training=true2020.09 | 0.725 | 0.89 | 0.984 | |
| SA-BERT+HCLTraining=Hierarchical Curriculum Learning2020.12 | 0.721 | 0.896 | 0.993 | |
| BERT-VFTModel Type=PrLM-based2022.03 | 0.717 | 0.884 | 0.986 | |
| BERT + SPIDERTraining Mode=Domain Adaptive Post-training2021.05 | 0.708 | 0.853 | 0.986 | |
| UMSELECTRA+Domain-specific post-training=true2020.09 | 0.707 | 0.853 | 0.974 | |
| UMS-ELECTRA+Model Type=PrLMs-based2022.03 | 0.707 | 0.853 | 0.974 | |
| SA-BERTDomain-specific post-training=false2020.09 | 0.704 | 0.879 | 0.985 | |
| SA-BERTTraining Mode=Baseline2021.05 | 0.704 | 0.879 | 0.985 | |
| SA-BERT2020.12 | 0.704 | 0.879 | 0.985 | |
| SA-BERTModel Type=PrLM-based2022.03 | 0.704 | 0.879 | 0.985 | |
| UMSBERTDomain-specific post-training=false2020.09 | 0.674 | 0.861 | 0.98 | |
| ELECTRA+Domain-specific post-training=true2020.09 | 0.673 | 0.835 | 0.974 | |
| BERTTraining Mode=Domain Adaptive Post-training2021.05 | 0.664 | 0.848 | 0.976 | |
| BERT-SS-DADomain-specific post-training=false2020.09 | 0.648 | 0.843 | 0.98 | |
| UMSELECTRADomain-specific post-training=false2020.09 | 0.648 | 0.831 | 0.974 | |
| MSN+HCLTraining=Hierarchical Curriculum Learning2020.12 | 0.642 | 0.814 | 0.968 | |
| BERT (ours)Domain-specific post-training=false2020.09 | 0.641 | 0.824 | 0.973 | |
| MDFNModel Type=PrLMs-based2022.03 | 0.639 | 0.829 | 0.971 | |
| BERT + SPIDERTraining Mode=Multi-task Fine-tuning2021.05 | 0.626 | 0.827 | 0.971 | |
| IMNModel Type=non PrLM-based2022.03 | 0.621 | 0.797 | 0.964 | |
| BERTTraining Mode=Multi-task Fine-tuning2021.05 | 0.617 | 0.811 | 0.97 | |
| G-MSNtraining protocol=grayscale enhanced training2020.04 | 0.613 | 0.786 | 0.964 | |
| BERT (Gu et al. 2020)Domain-specific post-training=false2020.09 | 0.61 | 0.814 | 0.973 | |
| BERTModel Type=PrLM-based2022.03 | 0.61 | 0.814 | 0.973 | |
| ELECTRADomain-specific post-training=false2020.09 | 0.609 | 0.804 | 0.965 | |
| ELECTRAModel Type=PrLMs-based2022.03 | 0.607 | 0.813 | 0.96 | |
| MSN2020.04 | 0.606 | 0.77 | 0.937 | |
| MSNDomain-specific post-training=false2020.09 | 0.606 | 0.77 | 0.937 | |
| MSNTraining Mode=Baseline2021.05 | 0.606 | 0.77 | 0.937 | |
| MSN2020.12 | 0.606 | 0.77 | 0.937 | |
| MSNModel Type=non PrLM-based2022.03 | 0.606 | 0.77 | 0.937 | |
| G-IOItraining protocol=grayscale enhanced training2020.04 | 0.579 | 0.772 | 0.955 | |
| G-DAMtraining protocol=grayscale enhanced training2020.04 | 0.564 | 0.769 | 0.948 | |
| IOI2020.04 | 0.563 | 0.768 | 0.95 | |
| IoIDomain-specific post-training=false2020.09 | 0.563 | 0.768 | 0.95 | |
| IOI2020.12 | 0.563 | 0.768 | 0.95 | |
| DAM2020.04 | 0.526 | 0.727 | 0.933 | |
| DAMDomain-specific post-training=false2020.09 | 0.526 | 0.727 | 0.933 | |
| DAM2020.12 | 0.526 | 0.727 | 0.933 | |
| DAMModel Type=non PrLM-based2022.03 | 0.526 | 0.727 | 0.933 | |
| SMN+HCLTraining=Hierarchical Curriculum Learning2020.12 | 0.507 | 0.723 | 0.935 | |
| G-SMNtraining protocol=grayscale enhanced training2020.04 | 0.504 | 0.713 | 0.926 | |
| DUA2020.04 | 0.501 | 0.7 | 0.921 | |
| DUADomain-specific post-training=false2020.09 | 0.501 | 0.7 | 0.921 | |
| DUATraining Mode=Baseline2021.05 | 0.501 | 0.7 | 0.921 | |
| DUA2020.12 | 0.501 | 0.7 | 0.921 | |
| DUAModel Type=non PrLM-based2022.03 | 0.501 | 0.7 | 0.921 | |
| SMN2020.04 | 0.453 | 0.654 | 0.886 | |
| SMNDomain-specific post-training=false2020.09 | 0.453 | 0.654 | 0.886 | |
| SMNTraining Mode=Baseline2021.05 | 0.453 | 0.654 | 0.886 | |
| SMN2020.12 | 0.453 | 0.654 | 0.886 | |
| SMNModel Type=non PrLM-based2022.03 | 0.453 | 0.654 | 0.886 | |
| Multi-View2020.04 | 0.421 | 0.601 | 0.861 | |
| Multi-ViewDomain-specific post-training=false2020.09 | 0.421 | 0.601 | 0.861 | |
| Multi-View2020.12 | 0.421 | 0.601 | 0.861 | |
| MV-LSTM2020.04 | 0.412 | 0.591 | 0.857 | |
| MV-LSTMDomain-specific post-training=false2020.09 | 0.412 | 0.591 | 0.857 | |
| MV-LSTM2020.12 | 0.412 | 0.591 | 0.857 | |
| Match-LSTM2020.04 | 0.41 | 0.59 | 0.858 | |
| Match-LSTMDomain-specific post-training=false2020.09 | 0.41 | 0.59 | 0.858 | |
| Match-LSTM2020.12 | 0.41 | 0.59 | 0.858 | |
| DL2R2020.04 | 0.399 | 0.571 | 0.842 | |
| DL2RDomain-specific post-training=false2020.09 | 0.399 | 0.571 | 0.842 | |
| DL2R2020.12 | 0.399 | 0.571 | 0.842 | |
| LSTM2020.04 | 0.365 | 0.536 | 0.828 | |
| LSTMDomain-specific post-training=false2020.09 | 0.365 | 0.536 | 0.828 | |
| BiLSTMDomain-specific post-training=false2020.09 | 0.365 | 0.536 | 0.825 | |
| LSTM2020.12 | 0.365 | 0.536 | 0.828 | |
| LSTMModel Type=non PrLM-based2022.03 | 0.365 | 0.536 | 0.828 | |
| BiLSTM2020.04 | 0.355 | 0.525 | 0.825 | |
| Bi-LSTM2020.12 | 0.355 | 0.525 | 0.825 | |
| CNN2020.04 | 0.328 | 0.515 | 0.792 | |
| CNNDomain-specific post-training=false2020.09 | 0.328 | 0.515 | 0.792 | |
| CNN2020.12 | 0.328 | 0.515 | 0.792 | |
| RNN2020.04 | 0.325 | 0.463 | 0.775 | |
| RNN2020.12 | 0.325 | 0.463 | 0.775 |