Answer Sentence Selection on ASNQ (test)
71.3P@1DeBERTaV3Base + SSP (ALL)
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| DeBERTaV3Base + SSP (ALL)fine-tuning context=true2023.09 | 71.3 | 75.1 | 80.5 | |
| ELECTRABase + SSP (DPC)fine-tuning context=true2023.09 | 70.5 | 74.4 | 79.5 | |
| ELECTRABase + SSP (DPC)CTX=true, Objective=Dynamic Paragraph-level Context2023.09 | 70.5 | 74.4 | 79.5 | |
| DeBERTaV3Basefine-tuning context=true2023.09 | 70.3 | 74.4 | 79.8 | |
| ELECTRABase + SSP (DSLC)fine-tuning context=true2023.09 | 70 | 74.3 | 79.4 | |
| ELECTRABase + SSP (DSLC)CTX=true, Objective=Dynamic Sentence-level Local Context2023.09 | 70 | 74.3 | 79.4 | |
| ELECTRABase + SSP (SDC)fine-tuning context=true2023.09 | 69.6 | 74.7 | 79.5 | |
| ELECTRABase + SSP (SDC)CTX=true, Objective=Static Document-level Context2023.09 | 69.6 | 74.7 | 79.5 | |
| RoBERTaBase + SSP (DPC)fine-tuning context=true2023.09 | 69.2 | 73.2 | 78.5 | |
| RoBERTaBase + SSP (DPC)CTX=true, Objective=Dynamic Paragraph-level Context2023.09 | 69.2 | 73.2 | 78.5 | |
| RoBERTaBase + SSP (DSLC)fine-tuning context=true2023.09 | 69 | 72.9 | 78.4 | |
| RoBERTaBase + SSP (DSLC)CTX=true, Objective=Dynamic Sentence-level Local Context2023.09 | 69 | 72.9 | 78.4 | |
| ELECTRABase + SSP (ALL)fine-tuning context=true2023.09 | 68.8 | 73.8 | 78.7 | |
| ELECTRABase + SSP (ALL)CTX=true2023.09 | 68.8 | 73.8 | 78.7 | |
| RoBERTaBase + SSP (SDC)fine-tuning context=true2023.09 | 68.7 | 73.1 | 78.4 | |
| RoBERTaBase + SSP (SDC)CTX=true, Objective=Static Document-level Context2023.09 | 68.7 | 73.1 | 78.4 | |
| RoBERTaBase + SSP (ALL)fine-tuning context=true2023.09 | 68.2 | 72.9 | 77.9 | |
| RoBERTaBase + SSP (ALL)CTX=true2023.09 | 68.2 | 72.9 | 77.9 | |
| ELECTRABasefine-tuning context=true2023.09 | 68.1 | 72.3 | 77.9 | |
| ELECTRABaseCTX=true, Source=Lauriola et al., 20212023.09 | 68.1 | 72.3 | 77.9 | |
| DeBERTaBase + SSP (ALL)fine-tuning context=true2023.09 | 67.7 | 72.2 | 77.9 | |
| RoBERTaBasefine-tuning context=true2023.09 | 67.6 | 71.6 | 77.3 | |
| RoBERTaBaseCTX=true, Source=Lauriola et al., 20212023.09 | 67.6 | 71.6 | 77.3 | |
| DeBERTaV3Basefine-tuning context=false2023.09 | 67.5 | 70.8 | 77 | |
| DeBERTaBasefine-tuning context=true2023.09 | 66.2 | 71.1 | 76.2 | |
| DeBERTaBasefine-tuning context=false2023.09 | 65.3 | 69.1 | 75.3 | |
| ELECTRA + SSPBase Model=ELECTRA-Base, Pre-training objective=SSP2022.05 | 65.3 | 69.7 | 75.7 | |
| ELECTRA + PSDBase Model=ELECTRA-Base, Pre-training objective=PSD2022.05 | 65.3 | 68.9 | 75.1 | |
| ELECTRABasefine-tuning context=false2023.09 | 65 | 69.3 | 75.1 | |
| ELECTRABaseCTX=false, Source=Clark et al., 20202023.09 | 65 | 69.3 | 75.1 | |
| ELECTRA + SPBase Model=ELECTRA-Base, Pre-training objective=SP2022.05 | 65 | 69 | 75.1 | |
| ELECTRA + AllBase Model=ELECTRA-Base, Pre-training objective=SSP+SP+PSD2022.05 | 65 | 69.3 | 75.2 | |
| RoBERTa + SSPBase Model=RoBERTa-Base, Pre-training objective=SSP2022.05 | 64.1 | 68.1 | 74.5 | |
| RoBERTa + SPBase Model=RoBERTa-Base, Pre-training objective=SP2022.05 | 64.1 | 68.3 | 74.5 | |
| RoBERTa + AllBase Model=RoBERTa-Base, Pre-training objective=SSP+SP+PSD2022.05 | 63.9 | 68 | 74.1 | |
| RoBERTaBasefine-tuning context=false2023.09 | 63.5 | 68.2 | 74.2 | |
| RoBERTaBaseCTX=false, Source=Liu et al., 2019b2023.09 | 63.5 | 68.2 | 74.2 | |
| Joint MSPP IEkArchitecture=RoBERTa-Base, Pre-training task=MSPP, Evaluation Protocol=Joint model, Prediction Head=IEk, Fine-tuning=FT IEk2022.05 | 63 | 67.2 | 73.7 | |
| Joint MSPP AEkArchitecture=RoBERTa-Base, Pre-training task=MSPP, Evaluation Protocol=Joint model, Prediction Head=AEk, Fine-tuning=FT AEk2022.05 | 63 | 67.3 | 73.7 | |
| RoBERTa + PSDBase Model=RoBERTa-Base, Pre-training objective=PSD2022.05 | 62.6 | 67.7 | 73.7 | |
| ELECTRA-BaseBase Model=ELECTRA-Base2022.05 | 62.4 | 67.5 | 73.6 | |
| Pairwise RoBERTa-BaseArchitecture=RoBERTa-Base, Evaluation Protocol=Pairwise cross-encoder2022.05 | 61.8 | 66.9 | 73.1 | |
| RoBERTa-BaseBase Model=RoBERTa-Base2022.05 | 61.8 | 66.9 | 73.1 | |
| Joint RoBERTa-BaseArchitecture=RoBERTa-Base, Evaluation Protocol=Joint model, Prediction Head=AEk, Fine-tuning=FT AEk2022.05 | 3.6 | 8 | 10.2 | |
| Joint RoBERTa-BaseArchitecture=RoBERTa-Base, Evaluation Protocol=Joint model, Prediction Head=IEk, Fine-tuning=FT IEk2022.05 | 3.4 | 8 | 10 |