Logical Reasoning on ReClor 1.0 (test-H)
72.9AccuracyMERIT
Evaluation Results
| Method | Links | |
|---|---|---|
| MERITBase Architecture=DeBERTa-v2-xxlarge2022.12 | 72.9 | |
| APOLLOBase Architecture=DeBERTa-v2-xxlarge2022.12 | 69.6 | |
| DeBERTa-v2-xxlargeBase Architecture=DeBERTa-v2-xxlarge2022.12 | 68.4 | |
| APOLLOBase Architecture=DeBERTa-v32022.12 | 65.7 | |
| DeBERTa-v3Base Architecture=DeBERTa-v32022.12 | 64 | |
| Human Performance2022.05 | 0.672 | |
| LogiformerBackbone=RoBERTa-Large2022.05 | 0.5125 | |
| FocalReasonerBackbone=RoBERTa-Large2022.05 | 0.4464 | |
| DAGNBackbone=RoBERTa-Large2022.05 | 0.4446 | |
| XLNet-LargeBackbone=XLNet-Large2022.05 | 0.405 | |
| RoBERTa-LargeBackbone=RoBERTa-Large2022.05 | 0.4 | |
| BERT-LargeBackbone=BERT-Large2022.05 | 0.323 | |
| Random2022.05 | 0.25 |