Logical Reasoning on LogiQA (dev)
47.3AccuracyFOCAL REASONER
Evaluation results
| Method | Links | |
|---|---|---|
| FOCAL REASONERBackbone=DeBERTa, Data Augmentation=false2021.05 | 47.3 | |
| LReasonerBackbone=DeBERTa2021.05 | 45.8 | |
| HGNBackbone=DeBERTa2021.05 | 44.9 | |
| IDOLBackbone=ALBERT2023.06 | 44.7 | |
| DeBERTaBackbone=DeBERTa2021.05 | 44.4 | |
| MERITBackbone=ALBERT2023.06 | 43.9 | |
| IDOLBackbone=RoBERTa2023.06 | 42.5 | |
| MERITBase model=RoBERTa-large2022.12 | 42.4 | |
| MERITBackbone=RoBERTa2023.06 | 42.4 | |
| LogiformerBackbone=RoBERTa2023.06 | 42.2 | |
| APOLLOBase model=RoBERTa-large2022.12 | 41.6 | |
| LReasonerBackbone=ALBERT2023.06 | 41.6 | |
| ALBERTBackbone=ALBERT2023.06 | 41.2 | |
| Focal Reasoner2022.03 | 41.01 | |
| FOCAL REASONERBase model=RoBERTa-large2022.12 | 41 | |
| FOCAL REASONERBackbone=RoBERTa, Data Augmentation=false2021.05 | 41 | |
| HGNBackbone=RoBERTa2021.05 | 40.1 | |
| MERITBackbone=RoBERTa, Data Augmentation=true2021.05 | 40 | |
| AdaLoGN2022.03 | 39.94 | |
| AdaLoGNBackbone=RoBERTa2023.06 | 39.9 | |
| LReasonerdata_augmentation=true2022.03 | 38.1 | |
| LRReasonerBase model=RoBERTa-large2022.12 | 38.1 | |
| LReasonerBackbone=RoBERTa, Data Augmentation=true2021.05 | 38.1 | |
| LReasonerBackbone=RoBERTa2023.06 | 38.1 | |
| RoBERTaBackbone=RoBERTa2023.06 | 37 | |
| DAGNBackbone=RoBERTa, Data Augmentation=true2021.05 | 36.9 | |
| IDOLBackbone=BERT2023.06 | 36.9 | |
| DAGN (Aug)Backbone=RoBERTa-Large, Graph Feature Augmentation=true2021.03 | 36.87 | |
| DAGN2022.03 | 36.87 | |
| DAGNBase model=RoBERTa-large2022.12 | 35.5 | |
| DAGNBackbone=RoBERTa, Data Augmentation=false2021.05 | 35.5 | |
| DAGNBackbone=RoBERTa2023.06 | 35.5 | |
| DAGNBackbone=RoBERTa-Large2021.03 | 35.48 | |
| BERTBackbone=BERT2023.06 | 35.3 | |
| RoBERTa-Large2021.03 | 35.02 | |
| RoBERTamodel_size=Large2022.03 | 35.02 | |
| RoBERTaBase model=RoBERTa-large2022.12 | 35 | |
| RoBERTaBackbone=RoBERTa2021.05 | 35 | |
| BERT-Large2021.03 | 34.1 | |
| BERTmodel_size=Large2022.03 | 34.1 |