Commonsense Reasoning on aNLI
87.3AccuracyRAINBOW
Evaluation Results
| Method | Links | |
|---|---|---|
| RAINBOWknowledge-augmented=true, plausibility annotations=with2023.05 | 87.3 | |
| RoBERTaplausibility annotations=with2023.05 | 85.6 | |
| DRAGON2022.10 | 84 | |
| GreaseLM2022.10 | 83.3 | |
| QAGNN2022.10 | 83 | |
| RoBERTa2022.10 | 82.7 | |
| LiPoRplausibility annotations=without2023.05 | 71.56 | |
| KDDC-ATOMIC (N)knowledge-augmented=true, plausibility annotations=without2023.05 | 70.8 | |
| KDDC-CSKG (N)knowledge-augmented=true, plausibility annotations=without2023.05 | 70.5 | |
| KDDC-CWWV (N)knowledge-augmented=true, plausibility annotations=without2023.05 | 70 | |
| ZS GPT3setting=zero-shot, plausibility annotations=without2023.05 | 67.54 | |
| Previous Bestplausibility annotations=without2023.05 | 65.5 | |
| CKT-baseBackbone=T5-base, Method=CKT2023.06 | 64.37 | |
| CALMBackbone=T5-base2023.06 | 63.2 | |
| KnowBERTBackbone=KnowBERT2023.06 | 63.18 | |
| ERNIE-baseBackbone=ERNIE-base2023.06 | 63.04 | |
| BERT-baseBackbone=BERT-base2023.06 | 61.88 | |
| T5-base + SSMBackbone=T5-base, Augmentation=SSM2023.06 | 61.57 | |
| T5-base + TIBackbone=T5-base, Augmentation=TI2023.06 | 61.42 | |
| T5-baseBackbone=T5-base2023.06 | 61.1 | |
| T5-base + KDBackbone=T5-base, Augmentation=KD2023.06 | 60.94 | |
| T5-base + CSKG (Rule)Backbone=T5-base, Augmentation=CSKG (Rule)2023.06 | 60.15 | |
| CKT w/ GPT-2Backbone=GPT-2, Method=CKT2023.06 | 60.14 | |
| T5-base + CSKG (TI)Backbone=T5-base, Augmentation=CSKG (TI)2023.06 | 59.92 | |
| COMETBackbone=BART2023.06 | 57.63 | |
| ZS GPT-NEOsetting=zero-shot, plausibility annotations=without2023.05 | 57.47 | |
| Tuned BARTsetting=tuned, plausibility annotations=without2023.05 | 57.4 | |
| ZS BARTsetting=zero-shot, plausibility annotations=without2023.05 | 50.96 | |
| MDM-Prime-v2Zero-shot=true, Number of Parameters=1.1B2026.03 | 34.24 | |
| OPTZero-shot=true, Number of Parameters=1.3B2026.03 | 33.86 | |
| BloomZero-shot=true, Number of Parameters=1.1B2026.03 | 33.35 | |
| PythiaZero-shot=true, Number of Parameters=1.4B2026.03 | 33.29 | |
| TinyLLaMAZero-shot=true, Number of Parameters=1.1B2026.03 | 33.25 | |
| GPT-NeoZero-shot=true, Number of Parameters=1.3B2026.03 | 33.13 | |
| SMDMZero-shot=true, Number of Parameters=1.1B2026.03 | 32.81 |