Commonsense Question Answering on CommonsenseQA v1.0 (dev)
79.3AccuracyOur Model
Evaluation Results
| Method | Links | |
|---|---|---|
| Our ModelGroup=42019.09 | 79.3 | |
| ROBERTa + IRGroup=42019.09 | 78.9 | |
| ROBERTaGroup=2, Mode=single2019.09 | 78.5 | |
| ROBERTa + KEGroup=42019.09 | 77.5 | |
| ROBERTa + CSPTGroup=3/42019.09 | 76.2 | |
| DREAMGroup=42019.09 | 73 | |
| GPT-3Strategy=Direct, Evaluation Protocol=Finetuned, Train Data Used (%)=1002022.03 | 73 | |
| STaRRationalization=With, Backbone=GPT-J, Train Data Used (%)=86.72022.03 | 72.5 | |
| BERT + OMCSGroup=42019.09 | 68.8 | |
| STaRRationalization=Without, Backbone=GPT-J, Train Data Used (%)=69.72022.03 | 68.8 | |
| ELABORAnswer predictor=T5-large, Elaboration model=GPT2-large2022.09 | 67.32 | |
| GPT-3Answer predictor=T5-large, Knowledge source=GPT-32022.09 | 67.23 | |
| pipelineAnswer predictor=T5-large, Elaboration model=GPT2-large2022.09 | 66.42 | |
| COMETAnswer predictor=T5-large, Knowledge source=COMET2022.09 | 66.34 | |
| scratchAnswer predictor=T5-large, Elaboration model=GPT2-large2022.09 | 65.36 | |
| vanillaAnswer predictor=T5-large, Knowledge source=None2022.09 | 65.19 | |
| selftalkAnswer predictor=T5-large, Knowledge source=selftalk2022.09 | 65.03 | |
| WikipediaAnswer predictor=T5-large, Knowledge source=Wikipedia2022.09 | 63.14 | |
| GPT-JStrategy=Direct, Evaluation Protocol=Finetuned, Train Data Used (%)=1002022.03 | 60 | |
| LaMDAModel Size=137B, Strategy=Chain-of-Thought (CoT), Evaluation Protocol=Few-shot, Train Data Used (%)=~02022.03 | 55.6 | |
| GPT-JStrategy=Chain-of-Thought (CoT), Evaluation Protocol=Few-shot, Train Data Used (%)=~02022.03 | 36.6 | |
| GPT-JStrategy=Direct, Evaluation Protocol=Few-shot, Train Data Used (%)=~02022.03 | 20.9 |