Answer Sentence Selection on TREC-QA
92.6P@1TANDA ELECTRA_Base (ours)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| TANDA ELECTRA_Base (ours)backbone=ELECTRA_Base, training=TANDA2023.09 | 92.6 | 91.6 | 95.5 | |
| ELECTRA_Base + ALLbackbone=ELECTRA_Base, objective=SSP + SP + PSD2023.09 | 92.6 | 90.4 | 95.5 | |
| Joint MSPP IEkArchitecture=Joint model, Pre-training Task=MSPP, Pre-training Head=IEk, Fine-tuning Head=IEk, Protocol=Re-ranking top-52022.05 | 92.2 | — | — | |
| ROBERTaBase Fixed (IEk) + MSPPBackbone=RoBERTa-base, Architecture Type=Fixed, Prediction Head=IEk, Pre-training Objective=MSPP2023.09 | 91.7 | 91.1 | 95.2 | |
| Joint MSPP AEkArchitecture=Joint model, Pre-training Task=MSPP, Pre-training Head=AEk, Fine-tuning Head=AEk, Protocol=Re-ranking top-52022.05 | 91.2 | — | — | |
| ROBERTaBase Flexible (REk) + MSPPBackbone=RoBERTa-base, Architecture Type=Flexible, Prediction Head=REk, Pre-training Objective=MSPP2023.09 | 91.2 | 91 | 95 | |
| ELECTRA_Base + SPbackbone=ELECTRA_Base, objective=Sentences in Paragraph2023.09 | 91.2 | 90.3 | 94.6 | |
| RoBERTa_Base + SPbackbone=RoBERTa_Base, objective=Sentences in Paragraph2023.09 | 90.9 | 90.1 | 94.7 | |
| RoBERTa_Base + PSDbackbone=RoBERTa_Base, objective=Paragraphs in the Same Document2023.09 | 90.3 | 90.3 | 95.1 | |
| ELECTRA_Basebackbone=ELECTRA_Base2023.09 | 90.3 | 89.9 | 94 | |
| RoBERTa_Basebackbone=RoBERTa_Base2023.09 | 90 | 89.7 | 94.4 | |
| TANDA RoBERTa_Base (ours)backbone=RoBERTa_Base, training=TANDA2023.09 | 89.7 | 90.1 | 94.1 | |
| ROBERTaBase Fixed (AEk) + MSPPBackbone=RoBERTa-base, Architecture Type=Fixed, Prediction Head=AEk, Pre-training Objective=MSPP2023.09 | 88.7 | 90.1 | 93.6 | |
| RoBERTa_Base + SSPbackbone=RoBERTa_Base, objective=Sentences in the Same Paragraph2023.09 | 88.5 | 89.3 | 93.6 | |
| ELECTRA_Base + SSPbackbone=ELECTRA_Base, objective=Sentences in the Same Paragraph2023.09 | 88.5 | 89.6 | 93.5 | |
| Pairwise RoBERTa-BaseArchitecture=Pairwise Cross-encoder, Protocol=Re-ranking top-52022.05 | 87.9 | — | — | |
| ROBERTaBaseBackbone=RoBERTa-base, Reference=Liu et al., 2019b2023.09 | 87.9 | 89.3 | 93.1 | |
| RoBERTa_Base + ALLbackbone=RoBERTa_Base, objective=SSP + SP + PSD2023.09 | 87.9 | 89.3 | 93.4 | |
| ELECTRA_Base + PSDbackbone=ELECTRA_Base, objective=Paragraphs in the Same Document2023.09 | 85.9 | 87.9 | 92.2 | |
| ROBERTaBase Flexible (REk)Backbone=RoBERTa-base, Architecture Type=Flexible, Prediction Head=REk, Pre-training Objective=Standard2023.09 | 84.8 | 84.8 | 91.3 | |
| Joint RoBERTa-BaseArchitecture=Joint model, Fine-tuning Head=AEk, Protocol=Re-ranking top-52022.05 | 60.9 | — | — | |
| Joint RoBERTa-BaseArchitecture=Joint model, Fine-tuning Head=IEk, Protocol=Re-ranking top-52022.05 | 57.6 | — | — | |
| ROBERTaBase Fixed (IEk)Backbone=RoBERTa-base, Architecture Type=Fixed, Prediction Head=IEk, Pre-training Objective=Standard2023.09 | 30.9 | 41.9 | 50.8 | |
| ROBERTaBase Fixed (AEk)Backbone=RoBERTa-base, Architecture Type=Fixed, Prediction Head=AEk, Pre-training Objective=Standard2023.09 | 29.7 | 42.3 | 49.2 | |
| CT RoBERTa_Basebackbone=RoBERTa_Base2023.09 | — | 92.4 | 96.7 | |
| RLAS-BIABC2023.09 | — | 89.8 | 90.6 | |
| TANDA RoBERTa_Basebackbone=RoBERTa_Base, training=TANDA2023.09 | — | 91.4 | 95.2 | |
| TANDA ROBERTaBaseBackbone=RoBERTa-base, Reference=Garg et al., 20192023.09 | — | 91.4 | 95.2 |