Question Answering on DuoRC MRQA out-of-domain evaluation
35.51EMFull-FT
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Full-FTEvaluation Protocol=Fine-tuning2025.10 | 35.51 | 48.82 | |
| SACCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 18.99 | 28.94 | |
| Lingua-2Compression Ratio=5x, Evaluation Protocol=Fine-tuning2025.10 | 18.32 | 27.07 | |
| EPLCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 16.39 | 25.7 | |
| BeaconCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 14.99 | 33.78 | |
| DASTCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 14.26 | 21.56 | |
| 500xCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 12.72 | 20.53 | |
| ICAECompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 9.06 | 13.78 | |
| RGXSource Domain=SQuAD_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=true, Cooperative self-training (CST)=true2021.03 | 0.574 | 0.662 | |
| RGX w/o MMISource Domain=SQuAD_wiki, Method=Extraction, Maximum Mutual Information (MMI)=false, EM selection (difficult questions)=true, Cooperative self-training (CST)=true2021.03 | 0.562 | 0.657 | |
| RGX w/o EMSource Domain=SQuAD_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=false, Cooperative self-training (CST)=true2021.03 | 0.556 | 0.649 | |
| RGX w/o CSTSource Domain=SQuAD_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=true, Cooperative self-training (CST)=false2021.03 | 0.539 | 0.653 | |
| QAGen2SSource Domain=SQuAD_wiki, Method=Extraction2021.03 | 0.533 | 0.646 | |
| ELECTRASource Domain=SQuAD_wiki, Method=Extraction2021.03 | 0.531 | 0.642 | |
| SynQASource Domain=SQuAD_wiki, Method=Extraction2021.03 | 0.517 | 0.621 | |
| T5 + RGXSource Domain=SQuAD_wiki, Method=Prompt Tuning + Seq2seq Generation2021.03 | 0.495 | 0.662 | |
| T5Source Domain=SQuAD_wiki, Method=Prompt Tuning + Seq2seq Generation2021.03 | 0.482 | 0.652 | |
| RGXSource Domain=NaturalQuestions_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=true, Cooperative self-training (CST)=true2021.03 | 0.478 | 0.584 | |
| RGX w/o MMISource Domain=NaturalQuestions_wiki, Method=Extraction, Maximum Mutual Information (MMI)=false, EM selection (difficult questions)=true, Cooperative self-training (CST)=true2021.03 | 0.469 | 0.575 | |
| RGX w/o EMSource Domain=NaturalQuestions_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=false, Cooperative self-training (CST)=true2021.03 | 0.466 | 0.569 | |
| RGX w/o CSTSource Domain=NaturalQuestions_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=true, Cooperative self-training (CST)=false2021.03 | 0.455 | 0.549 | |
| QAGen2SSource Domain=NaturalQuestions_wiki, Method=Extraction2021.03 | 0.438 | 0.532 | |
| ELECTRASource Domain=NaturalQuestions_wiki, Method=Extraction2021.03 | 0.4 | 0.485 |