Logical Reasoning on SemEval Task 11 (blind set)
90.05AccuracyNL
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| NLNotation=NL, Backbone=Flan-T5-large2026.06 | 90.05 | 9.57 | 26.81 | |
| NL-FOLNotation=NL-FOL, Backbone=Flan-T5-large2026.06 | 89 | 10.68 | 25.73 | |
| NL-FOL-CLIFNotation=NL-FOL-CLIF, Backbone=Flan-T5-large2026.06 | 84.29 | 10.68 | 24.37 | |
| NL-CLINGONotation=NL-CLINGO, Backbone=Flan-T5-large2026.06 | 84.29 | 10.7 | 24.36 | |
| NL-CLIFNotation=NL-CLIF, Backbone=Flan-T5-large2026.06 | 83.76 | 10.7 | 24.21 | |
| CLINGONotation=CLINGO, Backbone=Flan-T5-large2026.06 | 67.53 | 23.95 | 16.01 | |
| CGIFNotation=CGIF, Backbone=Flan-T5-large2026.06 | 66.49 | 28.12 | 15.21 | |
| CLIFNotation=CLIF, Backbone=Flan-T5-large2026.06 | 65.96 | 29.16 | 14.96 | |
| MINIFOL2Notation=MINIFOL2, Backbone=Flan-T5-large2026.06 | 64.92 | 23.95 | 15.39 | |
| FOL-CLIF-CLINGONotation=FOL-CLIF-CLINGO, Backbone=Flan-T5-large2026.06 | 64.39 | 27.08 | 14.85 | |
| FOLNotation=FOL, Backbone=Flan-T5-large2026.06 | 61.25 | 31.25 | 13.69 | |
| TFL+Notation=TFL+, Backbone=Flan-T5-large2026.06 | 59.16 | 5.89 | 20.18 |