List your company
ResearchFormerly WizWand

Commonsense Question Answering on CommonsenseQA v1.0 (dev)

79.3AccuracyOur Model
18.56434.33250.165.868Sep 9, 2019Mar 8, 2020Sep 6, 2020Mar 6, 2021Sep 4, 2021Mar 4, 2022Sep 2, 2022
Updated 8mo ago

Evaluation results

MethodLinks
Our ModelGroup=42019.0979.3
ROBERTa + IRGroup=42019.0978.9
ROBERTaGroup=2, Mode=single2019.0978.5
ROBERTa + KEGroup=42019.0977.5
ROBERTa + CSPTGroup=3/42019.0976.2
DREAMGroup=42019.0973
GPT-3Strategy=Direct, Evaluation Protocol=Finetuned, Train Data Used (%)=1002022.0373
STaRRationalization=With, Backbone=GPT-J, Train Data Used (%)=86.72022.0372.5
BERT + OMCSGroup=42019.0968.8
STaRRationalization=Without, Backbone=GPT-J, Train Data Used (%)=69.72022.0368.8
ELABORAnswer predictor=T5-large, Elaboration model=GPT2-large2022.0967.32
GPT-3Answer predictor=T5-large, Knowledge source=GPT-32022.0967.23
pipelineAnswer predictor=T5-large, Elaboration model=GPT2-large2022.0966.42
COMETAnswer predictor=T5-large, Knowledge source=COMET2022.0966.34
scratchAnswer predictor=T5-large, Elaboration model=GPT2-large2022.0965.36
vanillaAnswer predictor=T5-large, Knowledge source=None2022.0965.19
selftalkAnswer predictor=T5-large, Knowledge source=selftalk2022.0965.03
WikipediaAnswer predictor=T5-large, Knowledge source=Wikipedia2022.0963.14
GPT-JStrategy=Direct, Evaluation Protocol=Finetuned, Train Data Used (%)=1002022.0360
LaMDAModel Size=137B, Strategy=Chain-of-Thought (CoT), Evaluation Protocol=Few-shot, Train Data Used (%)=~02022.0355.6
GPT-JStrategy=Chain-of-Thought (CoT), Evaluation Protocol=Few-shot, Train Data Used (%)=~02022.0336.6
GPT-JStrategy=Direct, Evaluation Protocol=Few-shot, Train Data Used (%)=~02022.0320.9