Multiple Choice Question Answering on DREAM
98.77AccuracyQA+C
Evaluation Results
| Method | Links | |
|---|---|---|
| QA+CCoverage=20%2022.09 | 98.77 | |
| QA+E+CCoverage=20%2022.09 | 98.28 | |
| QA+ECoverage=20%2022.09 | 98.28 | |
| QACoverage=20%2022.09 | 98.28 | |
| QA+CCoverage=50%2022.09 | 96.86 | |
| CCoverage=20%2022.09 | 96.81 | |
| QACoverage=50%2022.09 | 96.67 | |
| E+CCoverage=20%2022.09 | 96.32 | |
| ECoverage=20%2022.09 | 96.32 | |
| QA+E+CCoverage=50%2022.09 | 95.1 | |
| UD+-XXLMode=Fully-supervised, Backbone=T5-XXL encoder, Model Parameters=11B2022.11 | 94.95 | |
| QA+ECoverage=50%2022.09 | 94.9 | |
| E+CCoverage=50%2022.09 | 93.63 | |
| ECoverage=50%2022.09 | 93.63 | |
| CCoverage=50%2022.09 | 93.63 | |
| Tay et al. (2022)Mode=Fully-supervised, Model Parameters=4x of UD (~44B)2022.11 | 91.8 | |
| QA+E+Cmodel=RoBERTa-RACE, entailment=true, contradiction=true2022.09 | 83.19 | |
| QA+Emodel=RoBERTa-RACE, entailment=true2022.09 | 82.85 | |
| QA+Cmodel=RoBERTa-RACE, contradiction=true2022.09 | 82.75 | |
| QAmodel=RoBERTa-RACE2022.09 | 82.56 | |
| SciQ-smallvariant=small2022.09 | 48.26 | |
| SciQ-basevariant=base2022.09 | 43.8 |