Commonsense Question Answering on SocialIQA (SIQA) (val)
70.7AccuracyChatGPT + Chain-of-thought
Evaluation results
| Method | Links | |
|---|---|---|
| ChatGPT + Chain-of-thoughtZero-shot=true, Variant=gpt-3.5-turbo, Prompting Strategy=Chain-of-thought2024.01 | 70.7 | |
| ChatGPT + Self-consistent chain-of-thoughtZero-shot=true, Variant=gpt-3.5-turbo, Prompting Strategy=Self-consistent chain-of-thought2024.01 | 69.7 | |
| ChatGPTZero-shot=true, Variant=gpt-3.5-turbo2024.01 | 69.5 | |
| GPT 3.5Zero-shot=true, Variant=text-davinci-0032024.01 | 68 | |
| DeBERTa-v3-L (CANDLE Distilled)Zero-shot=true, CSKB=CANDLE2024.01 | 65.9 | |
| STL-AdapterZero-shot=true, CSKB=ATOMIC2024.01 | 64.4 | |
| CAR-DeBERTa-v3-LZero-shot=true, CSKB=AbsATM2024.01 | 64 | |
| CAR-DeBERTa-v3-LZero-shot=true, CSKB=ATOMIC2024.01 | 63.8 | |
| DeBERTa-v3-L (MR)Zero-shot=true, CSKB=ATOMIC2024.01 | 62.1 | |
| DeBERTa-v3-L (MR)Zero-shot=true, CSKB=ATM10X2024.01 | 59.7 | |
| VERA-T5-xxl (CANDLE Distilled)Zero-shot=true, CSKB=CANDLE2024.01 | 59.4 | |
| VERA-T5-xxlZero-shot=true, CSKB=ATM10X2024.01 | 58.2 | |
| VERA-T5-xxlZero-shot=true, CSKB=AbsATM2024.01 | 58.1 | |
| VERA-T5-xxlZero-shot=true, CSKB=ATOMIC2024.01 | 57.7 | |
| GPT-4Zero-shot=true, Variant=gpt-42024.01 | 57 | |
| MICOZero-shot=true, CSKB=ATOMIC2024.01 | 56 | |
| LLAMA2Zero-shot=true, Model Scale=13B2024.01 | 50.3 | |
| COMET-DynGenZero-shot=true, CSKB=ATOMIC2024.01 | 50.1 | |
| SMLMZero-shot=true, CSKB=*2024.01 | 48.5 | |
| LLAMA2Zero-shot=true, Model Scale=7B2024.01 | 48.3 | |
| DeBERTa-v3-LZero-shot=true2024.01 | 47.8 | |
| ROBERTa-LZero-shot=true2024.01 | 47.3 | |
| Self-talkZero-shot=true2024.01 | 46.2 | |
| Mistral-v0.1Zero-shot=true, Model Scale=7B2024.01 | 42.9 |