Question Answering on DROP MRQA out-of-domain evaluation
64.9EMSynQA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SynQASource Domain=SQuAD_wiki, Method=Extraction2021.03 | 64.9 | 73 | |
| ELECTRASource Domain=SQuAD_wiki, Method=Extraction2021.03 | 48.3 | 60.8 | |
| RGXSource Domain=SQuAD_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=true, Cooperative self-training (CST)=true2021.03 | 47.6 | 60.9 | |
| RGX w/o MMISource Domain=SQuAD_wiki, Method=Extraction, Maximum Mutual Information (MMI)=false, EM selection (difficult questions)=true, Cooperative self-training (CST)=true2021.03 | 46.9 | 60.6 | |
| RGX w/o CSTSource Domain=SQuAD_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=true, Cooperative self-training (CST)=false2021.03 | 43 | 55.1 | |
| T5 + RGXSource Domain=SQuAD_wiki, Method=Prompt Tuning + Seq2seq Generation2021.03 | 42.9 | 53.8 | |
| QAGen2SSource Domain=SQuAD_wiki, Method=Extraction2021.03 | 42.2 | 54.5 | |
| RGX w/o EMSource Domain=SQuAD_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=false, Cooperative self-training (CST)=true2021.03 | 40.7 | 53.2 | |
| T5Source Domain=SQuAD_wiki, Method=Prompt Tuning + Seq2seq Generation2021.03 | 40.4 | 51.9 | |
| ELECTRASource Domain=NaturalQuestions_wiki, Method=Extraction2021.03 | 39.3 | 51.1 | |
| Full-FTEvaluation Protocol=Fine-tuning2025.10 | 34.46 | 44.67 | |
| RGXSource Domain=NaturalQuestions_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=true, Cooperative self-training (CST)=true2021.03 | 27.6 | 42.1 | |
| SACCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 27.48 | 36.72 | |
| RGX w/o MMISource Domain=NaturalQuestions_wiki, Method=Extraction, Maximum Mutual Information (MMI)=false, EM selection (difficult questions)=true, Cooperative self-training (CST)=true2021.03 | 27.1 | 41.7 | |
| RGX w/o EMSource Domain=NaturalQuestions_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=false, Cooperative self-training (CST)=true2021.03 | 26.1 | 40.9 | |
| RGX w/o CSTSource Domain=NaturalQuestions_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=true, Cooperative self-training (CST)=false2021.03 | 24.6 | 37.9 | |
| QAGen2SSource Domain=NaturalQuestions_wiki, Method=Extraction2021.03 | 24.2 | 37.1 | |
| 500xCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 23.55 | 33.46 | |
| DASTCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 22.42 | 31.9 | |
| EPLCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 22.29 | 32.16 | |
| ICAECompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 21.96 | 30.39 | |
| Lingua-2Compression Ratio=5x, Evaluation Protocol=Fine-tuning2025.10 | 18.83 | 27.28 | |
| BeaconCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 14.9 | 34.39 |