List your company
ResearchFormerly WizWand

Logical Reasoning on ReClor v1 (test)

78.1AccuracyMERIT
22.87637.21351.5565.887May 8, 2021Aug 14, 2021Nov 20, 2021Feb 27, 2022Jun 5, 2022Sep 11, 2022Dec 19, 2022
Updated 8mo ago

Evaluation results

MethodLinks
MERITBase Architecture=DeBERTa-v2-xxlarge2022.1278.1
APOLLOBase Architecture=DeBERTa-v2-xxlarge2022.1276.5
DeBERTa-v2-xxlargeBase Architecture=DeBERTa-v2-xxlarge2022.1275.3
APOLLOBase Architecture=DeBERTa-v32022.1272.8
DeBERTa-v3Base Architecture=DeBERTa-v32022.1271
LReasoner_ALBERTBackbone=ALBERT-xxlarge-v22021.0570.7
ALBERTBackbone=ALBERT-xxlarge-v22021.0566.5
LogiformerBackbone=RoBERTa-Large2022.0563.5
Human PerformanceEvaluator=Ten graduate students2021.0563
Human Performance2022.0563
LReasoner_RoBERTaBackbone=RoBERTa-large2021.0562.4
LReasonerBackbone=RoBERTa-Large, Model Type=Single Model2022.0562.4
FocalReasonerBackbone=RoBERTa-Large2022.0558.9
DAGNBackbone=RoBERTa-Large2022.0558.3
XLNetBackbone=XLNet-large2021.0556
XLNet-LargeBackbone=XLNet-Large2022.0556
RoBERTaBackbone=RoBERTa-large2021.0555.6
RoBERTa-LargeBackbone=RoBERTa-Large2022.0555.6
BERTBackbone=BERT-large2021.0549.8
BERT-LargeBackbone=BERT-Large2022.0549.8
GPT-2Backbone=GPT-22021.0547.2
GPTBackbone=GPT2021.0545.4
Random2022.0525