List your company
ResearchFormerly WizWand

Question Answering on CommonsenseQA IH (dev)

82.7AccuracyKnowGPT
17.49234.42151.3568.279Sep 4, 2019May 20, 2020Feb 4, 2021Oct 22, 2021Jul 9, 2022Mar 26, 2023Dec 11, 2023
Updated 8mo ago

Evaluation results

MethodLinks
KnowGPTCategory=Ours2023.1282.7
Ours2023.0579.8
GSC2023.0579.11
MindmapCategory=LLM + KG Prompting2023.1278.9
GreaseLM2023.0578.5
GreaseLMCategory=KG-enhanced LM2023.1278.5
GrapeQACategory=KG-enhanced LM2023.1278.2
JointLKCategory=KG-enhanced LM2023.1277.7
GPT-4Category=LLM + Zero-shot2023.1277.6
HamQACategory=KG-enhanced LM2023.1276.9
QA-GNN2023.0576.54
QA-GNNCategory=KG-enhanced LM2023.1276.5
CoKCategory=LLM + KG Prompting2023.1275.9
RoGCategory=LLM + KG Prompting2023.1275
RN2023.0574.57
MHGRNCategory=KG-enhanced LM2023.1274.5
Llama3Category=LLM + Zero-shot, Parameters=8b2023.1274.5
MHGRN2023.0574.45
GPT-3.5Category=LLM + Zero-shot, Implementation=gpt-3.5-turbo2023.1273.5
KagNet2023.0573.47
RoBerta-largeCategory=LM + Fine tuning2023.1273.1
RoBERTa-largeKnowledge Graph=without2023.0573.07
RGCN2023.0572.69
GconAttn2023.0572.61
BERT-LARGE-KAGNETTraining Data Ratio=100%, Backbone=BERT-Large2019.0962.35
Bert-largeCategory=LM + Fine tuning2023.1261.1
BERT-LARGE-FINETUNINGTraining Data Ratio=100%, Backbone=BERT-Large2019.0960.61
BERT-LARGE-KAGNETTraining Data Ratio=50%, Backbone=BERT-Large2019.0958.73
Bert-baseCategory=LM + Fine tuning2023.1257.3
Llama2Category=LLM + Zero-shot, Parameters=7b2023.1256.4
BERT-BASE-KAGNETTraining Data Ratio=100%, Backbone=BERT-Base2019.0955.57
BERT-LARGE-FINETUNINGTraining Data Ratio=50%, Backbone=BERT-Large2019.0955.45
GPT-3Category=LLM + Zero-shot, Implementation=text-davinci-0022023.1253.9
BERT-BASE-FINETUNINGTraining Data Ratio=100%, Backbone=BERT-Base2019.0953.48
BaichuanCategory=LLM + Zero-shot, Parameters=7B2023.1249.1
GPT-KAGNETTraining Data Ratio=100%, Backbone=GPT2019.0948.95
InternLMCategory=LLM + Zero-shot2023.1247.7
GPT-FINETUNINGTraining Data Ratio=100%, Backbone=GPT2019.0947.35
ChatGLMCategory=LLM + Zero-shot2023.1247.3
ChatGLM2Category=LLM + Zero-shot2023.1244
BERT-BASE-KAGNETTraining Data Ratio=50%, Backbone=BERT-Base2019.0940.32
BERT-BASE-FINETUNINGTraining Data Ratio=50%, Backbone=BERT-Base2019.0938.66
BERT-LARGE-KAGNETTraining Data Ratio=10%, Backbone=BERT-Large2019.0936.82
BERT-LARGE-FINETUNINGTraining Data Ratio=10%, Backbone=BERT-Large2019.0935.71
GPT-KAGNETTraining Data Ratio=50%, Backbone=GPT2019.0933.72
GPT-FINETUNINGTraining Data Ratio=50%, Backbone=GPT2019.0932.46
BERT-BASE-KAGNETTraining Data Ratio=10%, Backbone=BERT-Base2019.0931.05
BERT-BASE-FINETUNINGTraining Data Ratio=10%, Backbone=BERT-Base2019.0930.11
GPT-KAGNETTraining Data Ratio=10%, Backbone=GPT2019.0928.13
GPT-FINETUNINGTraining Data Ratio=10%, Backbone=GPT2019.0927.55
Random guessTraining Data Ratio=10%2019.0920
Random guessTraining Data Ratio=50%2019.0920
Random guessTraining Data Ratio=100%2019.0920