Answer Sentence Selection on TREC-QA (test)
94.88MAPASR
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ASRBackbone=RoBERTa Large, K=32021.07 | 94.88 | 98.16 | 0.9706 | |
| Our RerankerBackbone=RoBERTa Large2021.07 | 94.81 | 98.16 | 0.9706 | |
| RoBERTa-L TANDA (ASNQ → TREC-QA)Backbone=RoBERTa-Large, Training Strategy=TANDA (Transfer and Adapt), Source Dataset=ASNQ2019.11 | 94.3 | 97.4 | — | |
| Garg et al.Backbone=RoBERTa Large2021.07 | 94.3 | 97.4 | — | |
| KGATBackbone=RoBERTa Large, K=22021.07 | 94.07 | 97.43 | 0.9559 | |
| TANDA ELECTRABase Model=ELECTRA-Base, Intermediate transfer step=ASNQ2022.05 | 91.6 | 95.5 | 92.6 | |
| RoBERTa-B TANDA (ASNQ → TREC-QA)Backbone=RoBERTa-Base, Training Strategy=TANDA (Transfer and Adapt), Source Dataset=ASNQ2019.11 | 91.4 | 95.2 | — | |
| BERT-B TANDA (ASNQ → TREC-QA)Backbone=BERT-Base, Training Strategy=TANDA (Transfer and Adapt), Source Dataset=ASNQ2019.11 | 91.2 | 95.1 | — | |
| BERT-L TANDA (ASNQ → TREC-QA)Backbone=BERT-Large, Training Strategy=TANDA (Transfer and Adapt), Source Dataset=ASNQ2019.11 | 91.2 | 96.7 | — | |
| Joint MSPP IEkArchitecture=RoBERTa-Base, Pre-training task=MSPP, Evaluation Protocol=Joint model, Prediction Head=IEk, Fine-tuning=FT IEk2022.05 | 91.1 | 95.2 | 91.7 | |
| DeBERTaV3Basefine-tuning context=false2023.09 | 90.6 | 93.4 | 90.6 | |
| RoBERTaBase + SSP (DPC)fine-tuning context=true2023.09 | 90.5 | 95.3 | 91.2 | |
| RoBERTaBase + SSP (DSLC)fine-tuning context=true2023.09 | 90.5 | 95.7 | 92.2 | |
| RoBERTaBase + SSP (DPC)CTX=true2023.09 | 90.5 | 95.3 | 91.2 | |
| RoBERTaBase + SSP (DSLC)CTX=true2023.09 | 90.5 | 95.7 | 92.2 | |
| BERT-L FT TREC-QABackbone=BERT-Large, Training Strategy=Fine-tuned on TREC-QA2019.11 | 90.4 | 94.6 | — | |
| ELECTRA + AllBase Model=ELECTRA-Base, Pre-training objective=SSP+SP+PSD2022.05 | 90.4 | 95.5 | 92.6 | |
| RoBERTaBase + SSP (ALL)fine-tuning context=true2023.09 | 90.3 | 94.7 | 90.7 | |
| RoBERTaBase + SSP (ALL)CTX=true2023.09 | 90.3 | 94.7 | 90.7 | |
| RoBERTa + PSDBase Model=RoBERTa-Base, Pre-training objective=PSD2022.05 | 90.3 | 95.1 | 90.3 | |
| ELECTRA + SPBase Model=ELECTRA-Base, Pre-training objective=SP2022.05 | 90.3 | 94.6 | 91.2 | |
| DeBERTaBase + SSP (ALL)fine-tuning context=true2023.09 | 90.2 | 95 | 90.7 | |
| DeBERTaV3Base + SSP (ALL)fine-tuning context=true2023.09 | 90.2 | 94.3 | 91.5 | |
| Joint MSPP AEkArchitecture=RoBERTa-Base, Pre-training task=MSPP, Evaluation Protocol=Joint model, Prediction Head=AEk, Fine-tuning=FT AEk2022.05 | 90.1 | 93.6 | 88.7 | |
| ELECTRABase + SSP (SDC)fine-tuning context=true2023.09 | 90.1 | 94 | 89.7 | |
| ELECTRABase + SSP (SDC)CTX=true2023.09 | 90.1 | 94 | 89.7 | |
| RoBERTa + SPBase Model=RoBERTa-Base, Pre-training objective=SP2022.05 | 90.1 | 94.7 | 90.9 | |
| TANDA RoBERTaBase Model=RoBERTa-Base, Intermediate transfer step=ASNQ2022.05 | 90.1 | 94.1 | 89.7 | |
| RoBERTaBasefine-tuning context=false2023.09 | 89.9 | 93.7 | 89.2 | |
| ELECTRABasefine-tuning context=false2023.09 | 89.9 | 94 | 90.3 | |
| RoBERTaBaseCTX=false2023.09 | 89.9 | 93.7 | 89.2 | |
| ELECTRABaseCTX=false2023.09 | 89.9 | 94 | 90.3 | |
| ELECTRA-BaseBase Model=ELECTRA-Base2022.05 | 89.9 | 94 | 90.3 | |
| DeBERTaV3Basefine-tuning context=true2023.09 | 89.8 | 93.3 | 88.5 | |
| DeBERTaBasefine-tuning context=false2023.09 | 89.7 | 94.2 | 90.2 | |
| RoBERTa-BaseBase Model=RoBERTa-Base2022.05 | 89.7 | 94.4 | 90 | |
| ELECTRABase + SSP (ALL)fine-tuning context=true2023.09 | 89.6 | 93.5 | 88.7 | |
| DeBERTaBasefine-tuning context=true2023.09 | 89.6 | 94.1 | 89.7 | |
| ELECTRABase + SSP (ALL)CTX=true2023.09 | 89.6 | 93.5 | 88.7 | |
| ELECTRA + SSPBase Model=ELECTRA-Base, Pre-training objective=SSP2022.05 | 89.6 | 93.5 | 88.5 | |
| ELECTRABase + SSP (DPC)fine-tuning context=true2023.09 | 89.4 | 95.5 | 92.6 | |
| ELECTRABase + SSP (DPC)CTX=true2023.09 | 89.4 | 95.5 | 92.6 | |
| Pairwise RoBERTa-BaseArchitecture=RoBERTa-Base, Evaluation Protocol=Pairwise cross-encoder2022.05 | 89.3 | 93.1 | 87.9 | |
| RoBERTa + SSPBase Model=RoBERTa-Base, Pre-training objective=SSP2022.05 | 89.3 | 93.6 | 88.5 | |
| RoBERTa + AllBase Model=RoBERTa-Base, Pre-training objective=SSP+SP+PSD2022.05 | 89.3 | 93.4 | 87.9 | |
| ELECTRABase + SSP (DSLC)fine-tuning context=true2023.09 | 89.2 | 95 | 90.7 | |
| ELECTRABase + SSP (DSLC)CTX=true2023.09 | 89.2 | 95 | 90.7 | |
| RoBERTaBasefine-tuning context=true2023.09 | 88.2 | 93.5 | 88.2 | |
| RoBERTaBase + SSP (SDC)fine-tuning context=true2023.09 | 88.2 | 92.3 | 86.3 | |
| RoBERTaBaseCTX=true2023.09 | 88.2 | 93.5 | 88.2 | |
| RoBERTaBase + SSP (SDC)CTX=true2023.09 | 88.2 | 92.3 | 86.3 | |
| ELECTRABasefine-tuning context=true2023.09 | 88.1 | 92.9 | 87.3 | |
| ELECTRABaseCTX=true2023.09 | 88.1 | 92.9 | 87.3 | |
| RoBERTa-L FT ASNQBackbone=RoBERTa-Large, Training Strategy=Fine-tuned on ASNQ2019.11 | 88 | 92.8 | — | |
| ELECTRA + PSDBase Model=ELECTRA-Base, Pre-training objective=PSD2022.05 | 87.9 | 92.2 | 85.9 | |
| Comp-Agg + LM + LC + TL(QNLI)Backbone=Compare-Aggregate, Language Modeling (LM)=true, Latent Clustering (LC)=true, Transfer Learning Dataset=QNLI2019.11 | 87.5 | 94 | — | |
| Comp-Agg + LM + LCBackbone=Compare-Aggregate, Language Modeling (LM)=true, Latent Clustering (LC)=true2019.11 | 86.8 | 92.8 | — | |
| BERT-B FT TREC-QABackbone=BERT-Base, Training Strategy=Fine-tuned on TREC-QA2019.11 | 85.7 | 93.7 | — | |
| RoBERTa-B FT ASNQBackbone=RoBERTa-Base, Training Strategy=Fine-tuned on ASNQ2019.11 | 84.9 | 90.7 | — | |
| BERT-L FT ASNQBackbone=BERT-Large, Training Strategy=Fine-tuned on ASNQ2019.11 | 82.4 | 87.2 | — | |
| BERT-B FT ASNQBackbone=BERT-Base, Training Strategy=Fine-tuned on ASNQ2019.11 | 82.3 | 87.2 | — | |
| Joint RoBERTa-BaseArchitecture=RoBERTa-Base, Evaluation Protocol=Joint model, Prediction Head=AEk, Fine-tuning=FT AEk2022.05 | 42.3 | 49.2 | 29.7 | |
| Joint RoBERTa-BaseArchitecture=RoBERTa-Base, Evaluation Protocol=Joint model, Prediction Head=IEk, Fine-tuning=FT IEk2022.05 | 41.9 | 50.8 | 30.9 |