List your company
ResearchFormerly WizWand

Multiple Choice Question Answering on DREAM

98.77AccuracyQA+C
41.601256.443171.28586.1269Sep 28, 2022Oct 6, 2022Oct 14, 2022Oct 22, 2022Oct 30, 2022Nov 7, 2022Nov 15, 2022
Updated 8mo ago

Evaluation results

MethodLinks
QA+CCoverage=20%2022.0998.77
QA+E+CCoverage=20%2022.0998.28
QA+ECoverage=20%2022.0998.28
QACoverage=20%2022.0998.28
QA+CCoverage=50%2022.0996.86
CCoverage=20%2022.0996.81
QACoverage=50%2022.0996.67
E+CCoverage=20%2022.0996.32
ECoverage=20%2022.0996.32
QA+E+CCoverage=50%2022.0995.1
UD+-XXLMode=Fully-supervised, Backbone=T5-XXL encoder, Model Parameters=11B2022.1194.95
QA+ECoverage=50%2022.0994.9
E+CCoverage=50%2022.0993.63
ECoverage=50%2022.0993.63
CCoverage=50%2022.0993.63
Tay et al. (2022)Mode=Fully-supervised, Model Parameters=4x of UD (~44B)2022.1191.8
QA+E+Cmodel=RoBERTa-RACE, entailment=true, contradiction=true2022.0983.19
QA+Emodel=RoBERTa-RACE, entailment=true2022.0982.85
QA+Cmodel=RoBERTa-RACE, contradiction=true2022.0982.75
QAmodel=RoBERTa-RACE2022.0982.56
SciQ-smallvariant=small2022.0948.26
SciQ-basevariant=base2022.0943.8