Logical Reasoning on ReClor (test-e)
91.8AccuracyKnowledge Model
Evaluation Results
| Method | Links | |
|---|---|---|
| Knowledge ModelBackbone=DeBERTa2023.06 | 91.8 | |
| AMR-LEBackbone=DeBERTa2023.06 | 87.7 | |
| IDOLBackbone=DeBERTa2023.06 | 87.7 | |
| LReasonerBackbone=DeBERTa2023.06 | 87.1 | |
| MERITBackbone=DeBERTa2023.06 | 85.2 | |
| DeBERTaBackbone=DeBERTa2023.06 | 84 | |
| LRReasonerBase model=RoBERTa-large2022.12 | 81.4 | |
| PathReasoner2024.05 | 80.91 | |
| MERITBase model=RoBERTa-large2022.12 | 79.6 | |
| AdaLoGNCategory=Graph2024.05 | 79.32 | |
| MERITCategory=Sequence, Extra data=true2024.05 | 79.3 | |
| LogiformerCategory=Graph2024.05 | 79.09 | |
| FOCAL REASONERBase model=RoBERTa-large2022.12 | 77.1 | |
| FocalReasonerCategory=Graph2024.05 | 77.05 | |
| APOLLOBase model=RoBERTa-large2022.12 | 76.8 | |
| DAGNBase model=RoBERTa-large2022.12 | 76.1 | |
| DAGNCategory=Graph2024.05 | 75.91 | |
| XLNet-LargeCategory=Sequence2024.05 | 75.7 | |
| RoBERTaBase model=RoBERTa-large2022.12 | 75.5 | |
| ROBERTa-LargeCategory=Sequence2024.05 | 75.5 | |
| BERT-LargeCategory=Sequence2024.05 | 72 | |
| Human Performance2024.05 | 57.1 | |
| Random2024.05 | 25 |