Logical Reasoning on LogiQA 1.0 (test)
86AccuracyHuman Performance
Evaluation results
| Method | Links | |
|---|---|---|
| Human Performance2022.05 | 86 | |
| APOLLOBase Architecture=DeBERTa-v2-xxlarge2022.12 | 51 | |
| DeBERTa-v2-xxlargeBase Architecture=DeBERTa-v2-xxlarge2022.12 | 49.8 | |
| APOLLOBase Architecture=DeBERTa-v32022.12 | 44.4 | |
| LogiformerEncoder=RoBERTa-Large2022.05 | 42.55 | |
| FocalReasonerEncoder=RoBERTa-Large2022.05 | 40.25 | |
| DeBERTa-v3Base Architecture=DeBERTa-v32022.12 | 40.1 | |
| DAGNEncoder=RoBERTa-Large2022.05 | 39.32 | |
| RoBERTa-LargeEncoder=RoBERTa-Large2022.05 | 35.33 | |
| BERT-LargeEncoder=BERT-Large2022.05 | 31.03 | |
| Random2022.05 | 25 |