Commonsense Question Answering on Abductive NLI (aNLI) (val)
0.812AccuracyDeBERTa-v3-L (CANDLE Distilled)
Evaluation results
| Method | Links | |
|---|---|---|
| DeBERTa-v3-L (CANDLE Distilled)Zero-shot=true, CSKB=CANDLE2024.01 | 0.812 | |
| CAR-DeBERTa-v3-LZero-shot=true, CSKB=AbsATM2024.01 | 0.796 | |
| CAR-DeBERTa-v3-LZero-shot=true, CSKB=ATOMIC2024.01 | 0.789 | |
| DeBERTa-v3-L (MR)Zero-shot=true, CSKB=ATOMIC2024.01 | 0.76 | |
| DeBERTa-v3-L (MR)Zero-shot=true, CSKB=ATM10X2024.01 | 0.751 | |
| GPT-4Zero-shot=true, Variant=gpt-42024.01 | 0.75 | |
| VERA-T5-xxl (CANDLE Distilled)Zero-shot=true, CSKB=CANDLE2024.01 | 0.738 | |
| ChatGPT + Self-consistent chain-of-thoughtZero-shot=true, Variant=gpt-3.5-turbo, Prompting Strategy=Self-consistent chain-of-thought2024.01 | 0.732 | |
| VERA-T5-xxlZero-shot=true, CSKB=AbsATM2024.01 | 0.732 | |
| STL-AdapterZero-shot=true, CSKB=ATOMIC2024.01 | 0.713 | |
| VERA-T5-xxlZero-shot=true, CSKB=ATOMIC2024.01 | 0.712 | |
| ChatGPT + Chain-of-thoughtZero-shot=true, Variant=gpt-3.5-turbo, Prompting Strategy=Chain-of-thought2024.01 | 0.705 | |
| VERA-T5-xxlZero-shot=true, CSKB=ATM10X2024.01 | 0.703 | |
| ChatGPTZero-shot=true, Variant=gpt-3.5-turbo2024.01 | 0.693 | |
| ROBERTa-LZero-shot=true, Backbone=RoBERTa-Large2024.01 | 0.655 | |
| SMLMZero-shot=true, CSKB=*2024.01 | 0.653 | |
| GPT 3.5Zero-shot=true, Variant=text-davinci-0032024.01 | 0.618 | |
| DeBERTa-v3-LZero-shot=true, Backbone=DeBERTa-v3-Large2024.01 | 0.599 | |
| LLAMA2Zero-shot=true, Model Scale=7B2024.01 | 0.575 | |
| LLAMA2Zero-shot=true, Model Scale=13B2024.01 | 0.559 | |
| Mistral-v0.1Zero-shot=true, Model Scale=7B2024.01 | 0.51 |