List your company
ResearchFormerly WizWand

Question Answering on DuoRC MRQA out-of-domain evaluation

35.51EMFull-FT
-1.00448.475317.95527.4347Mar 12, 2021Dec 15, 2021Sep 20, 2022Jun 26, 2023Mar 31, 2024Jan 4, 2025Oct 10, 2025
Updated 7mo ago

Evaluation results

MethodLinks
Full-FTEvaluation Protocol=Fine-tuning2025.1035.5148.82
SACCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.1018.9928.94
Lingua-2Compression Ratio=5x, Evaluation Protocol=Fine-tuning2025.1018.3227.07
EPLCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.1016.3925.7
BeaconCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.1014.9933.78
DASTCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.1014.2621.56
500xCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.1012.7220.53
ICAECompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.109.0613.78
RGXSource Domain=SQuAD_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=true, Cooperative self-training (CST)=true2021.030.5740.662
RGX w/o MMISource Domain=SQuAD_wiki, Method=Extraction, Maximum Mutual Information (MMI)=false, EM selection (difficult questions)=true, Cooperative self-training (CST)=true2021.030.5620.657
RGX w/o EMSource Domain=SQuAD_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=false, Cooperative self-training (CST)=true2021.030.5560.649
RGX w/o CSTSource Domain=SQuAD_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=true, Cooperative self-training (CST)=false2021.030.5390.653
QAGen2SSource Domain=SQuAD_wiki, Method=Extraction2021.030.5330.646
ELECTRASource Domain=SQuAD_wiki, Method=Extraction2021.030.5310.642
SynQASource Domain=SQuAD_wiki, Method=Extraction2021.030.5170.621
T5 + RGXSource Domain=SQuAD_wiki, Method=Prompt Tuning + Seq2seq Generation2021.030.4950.662
T5Source Domain=SQuAD_wiki, Method=Prompt Tuning + Seq2seq Generation2021.030.4820.652
RGXSource Domain=NaturalQuestions_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=true, Cooperative self-training (CST)=true2021.030.4780.584
RGX w/o MMISource Domain=NaturalQuestions_wiki, Method=Extraction, Maximum Mutual Information (MMI)=false, EM selection (difficult questions)=true, Cooperative self-training (CST)=true2021.030.4690.575
RGX w/o EMSource Domain=NaturalQuestions_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=false, Cooperative self-training (CST)=true2021.030.4660.569
RGX w/o CSTSource Domain=NaturalQuestions_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=true, Cooperative self-training (CST)=false2021.030.4550.549
QAGen2SSource Domain=NaturalQuestions_wiki, Method=Extraction2021.030.4380.532
ELECTRASource Domain=NaturalQuestions_wiki, Method=Extraction2021.030.40.485