Question Answering on RACE MRQA out-of-domain evaluation
46.3EMRGX w/o MMI
Evaluation results
| Method | Links | ||
|---|---|---|---|
| RGX w/o MMISource Domain=SQuAD_wiki, Method=Extraction, Maximum Mutual Information (MMI)=false, EM selection (difficult questions)=true, Cooperative self-training (CST)=true2021.03 | 46.3 | 57.6 | |
| RGXSource Domain=SQuAD_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=true, Cooperative self-training (CST)=true2021.03 | 44.9 | 58.7 | |
| RGX w/o CSTSource Domain=SQuAD_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=true, Cooperative self-training (CST)=false2021.03 | 43.8 | 55.2 | |
| QAGen2SSource Domain=SQuAD_wiki, Method=Extraction2021.03 | 43.4 | 54.9 | |
| RGXSource Domain=NaturalQuestions_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=true, Cooperative self-training (CST)=true2021.03 | 40.3 | 52.4 | |
| SynQASource Domain=SQuAD_wiki, Method=Extraction2021.03 | 40.2 | 54.2 | |
| RGX w/o MMISource Domain=NaturalQuestions_wiki, Method=Extraction, Maximum Mutual Information (MMI)=false, EM selection (difficult questions)=true, Cooperative self-training (CST)=true2021.03 | 39.7 | 51.5 | |
| RGX w/o EMSource Domain=NaturalQuestions_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=false, Cooperative self-training (CST)=true2021.03 | 38.3 | 50.5 | |
| ELECTRASource Domain=SQuAD_wiki, Method=Extraction2021.03 | 38.3 | 52.5 | |
| RGX w/o EMSource Domain=SQuAD_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=false, Cooperative self-training (CST)=true2021.03 | 35.4 | 48.3 | |
| RGX w/o CSTSource Domain=NaturalQuestions_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=true, Cooperative self-training (CST)=false2021.03 | 35.1 | 47.2 | |
| QAGen2SSource Domain=NaturalQuestions_wiki, Method=Extraction2021.03 | 33.7 | 45.5 | |
| ELECTRASource Domain=NaturalQuestions_wiki, Method=Extraction2021.03 | 32.4 | 43.4 | |
| T5 + RGXSource Domain=SQuAD_wiki, Method=Prompt Tuning + Seq2seq Generation2021.03 | 15.5 | 55.1 | |
| T5Source Domain=SQuAD_wiki, Method=Prompt Tuning + Seq2seq Generation2021.03 | 15 | 53.1 | |
| Full-FTEvaluation Protocol=Fine-tuning2025.10 | 9.64 | 35.57 | |
| SACCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 4.9 | 23.35 | |
| Lingua-2Compression Ratio=5x, Evaluation Protocol=Fine-tuning2025.10 | 4.15 | 17.54 | |
| EPLCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 4.01 | 20.97 | |
| ICAECompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 3.71 | 15.21 | |
| BeaconCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 3.71 | 26.68 | |
| 500xCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 3.41 | 18.49 | |
| DASTCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 3.26 | 16.31 |