Question Answering on MRQA Average across 6 domains
56.8EMRGX
Evaluation results
| Method | Links | ||
|---|---|---|---|
| RGXSource Domain=SQuAD_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=true, Cooperative self-training (CST)=true2021.03 | 56.8 | 68.4 | |
| RGX w/o MMISource Domain=SQuAD_wiki, Method=Extraction, Maximum Mutual Information (MMI)=false, EM selection (difficult questions)=true, Cooperative self-training (CST)=true2021.03 | 56.3 | 67.7 | |
| SynQASource Domain=SQuAD_wiki, Method=Extraction2021.03 | 55.3 | 66.1 | |
| RGX w/o CSTSource Domain=SQuAD_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=true, Cooperative self-training (CST)=false2021.03 | 53.5 | 65 | |
| ELECTRASource Domain=SQuAD_wiki, Method=Extraction2021.03 | 53.4 | 65.4 | |
| QAGen2SSource Domain=SQuAD_wiki, Method=Extraction2021.03 | 52.8 | 64.5 | |
| RGX w/o EMSource Domain=SQuAD_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=false, Cooperative self-training (CST)=true2021.03 | 51.7 | 62.5 | |
| RGXSource Domain=NaturalQuestions_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=true, Cooperative self-training (CST)=true2021.03 | 48.7 | 61.9 | |
| RGX w/o MMISource Domain=NaturalQuestions_wiki, Method=Extraction, Maximum Mutual Information (MMI)=false, EM selection (difficult questions)=true, Cooperative self-training (CST)=true2021.03 | 46.8 | 61.2 | |
| RGX w/o EMSource Domain=NaturalQuestions_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=false, Cooperative self-training (CST)=true2021.03 | 46.8 | 60.4 | |
| T5 + RGXSource Domain=SQuAD_wiki, Method=Prompt Tuning + Seq2seq Generation2021.03 | 46.7 | 66.3 | |
| T5Source Domain=SQuAD_wiki, Method=Prompt Tuning + Seq2seq Generation2021.03 | 45.1 | 64.9 | |
| RGX w/o CSTSource Domain=NaturalQuestions_wiki, Method=Extraction, Maximum Mutual Information (MMI)=true, EM selection (difficult questions)=true, Cooperative self-training (CST)=false2021.03 | 44.2 | 57.6 | |
| QAGen2SSource Domain=NaturalQuestions_wiki, Method=Extraction2021.03 | 42.7 | 56 | |
| ELECTRASource Domain=NaturalQuestions_wiki, Method=Extraction2021.03 | 42.2 | 54.2 | |
| Full-FTEvaluation Protocol=Fine-tuning2025.10 | 36.87 | 52.51 | |
| SACCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 26.02 | 39.26 | |
| EPLCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 23.83 | 36.74 | |
| 500xCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 22.09 | 34.04 | |
| DASTCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 20.95 | 31.93 | |
| ICAECompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 20.18 | 30.81 | |
| Lingua-2Compression Ratio=5x, Evaluation Protocol=Fine-tuning2025.10 | 16.2 | 27.9 | |
| BeaconCompression Ratio=15x, Evaluation Protocol=Fine-tuning2025.10 | 13.02 | 36.91 |