List your company
ResearchFormerly WizWand

Question Answering on CommonsenseQA IH (test)

88.9AccuracyHuman Performance
17.24435.84754.4573.053Sep 4, 2019May 20, 2020Feb 4, 2021Oct 22, 2021Jul 9, 2022Mar 26, 2023Dec 11, 2023
Updated 8mo ago

Evaluation results

MethodLinks
Human PerformanceTraining Data Ratio=10%2019.0988.9
Human PerformanceTraining Data Ratio=50%2019.0988.9
Human PerformanceTraining Data Ratio=100%2019.0988.9
KnowGPTCategory=Ours2023.1281.8
GPT-4Category=LLM + Zero-shot2023.1278.6
MindmapCategory=LLM + KG Prompting2023.1278.4
Ours2023.0574.97
GrapeQACategory=KG-enhanced LM2023.1274.9
GSC2023.0574.48
JointLKCategory=KG-enhanced LM2023.1274.4
GreaseLM2023.0574.2
GreaseLMCategory=KG-enhanced LM2023.1274.2
SAFE2023.0574.03
HamQACategory=KG-enhanced LM2023.1273.9
CoKCategory=LLM + KG Prompting2023.1273.9
QA-GNN2023.0573.41
RoGCategory=LLM + KG Prompting2023.1273.4
QA-GNNCategory=KG-enhanced LM2023.1273.3
Llama3Category=LLM + Zero-shot, Parameters=8b2023.1272.3
MHGRNCategory=KG-enhanced LM2023.1271.3
MHGRN2023.0571.11
GPT-3.5Category=LLM + Zero-shot, Implementation=gpt-3.5-turbo2023.1271
RN2023.0569.08
KagNet2023.0569.01
RoBerta-largeCategory=LM + Fine tuning2023.1268.7
RoBERTa-largeKnowledge Graph=without2023.0568.69
GconAttn2023.0568.59
RGCN2023.0568.41
BERT-LARGE-KAGNETTraining Data Ratio=100%, Backbone=BERT-Large2019.0957.16
BERT-BASE-KAGNETTraining Data Ratio=100%, Backbone=BERT-Base2019.0956.19
BERT-LARGE-FINETUNINGTraining Data Ratio=100%, Backbone=BERT-Large2019.0955.84
Bert-largeCategory=LM + Fine tuning2023.1255.4
Llama2Category=LLM + Zero-shot, Parameters=7b2023.1254.6
Bert-baseCategory=LM + Fine tuning2023.1253.5
BERT-BASE-FINETUNINGTraining Data Ratio=100%, Backbone=BERT-Base2019.0953.26
GPT-3Category=LLM + Zero-shot, Implementation=text-davinci-0022023.1252
BERT-LARGE-KAGNETTraining Data Ratio=50%, Backbone=BERT-Large2019.0951.13
BERT-LARGE-FINETUNINGTraining Data Ratio=50%, Backbone=BERT-Large2019.0949.88
BaichuanCategory=LLM + Zero-shot, Parameters=7B2023.1247.6
ChatGLMCategory=LLM + Zero-shot2023.1246.9
GPT-KAGNETTraining Data Ratio=100%, Backbone=GPT2019.0946.79
GPT-FINETUNINGTraining Data Ratio=100%, Backbone=GPT2019.0945.58
InternLMCategory=LLM + Zero-shot2023.1245.4
ChatGLM2Category=LLM + Zero-shot2023.1242.5
BERT-BASE-KAGNETTraining Data Ratio=50%, Backbone=BERT-Base2019.0939.01
BERT-BASE-FINETUNINGTraining Data Ratio=50%, Backbone=BERT-Base2019.0936.83
BERT-LARGE-KAGNETTraining Data Ratio=10%, Backbone=BERT-Large2019.0933.91
BERT-LARGE-FINETUNINGTraining Data Ratio=10%, Backbone=BERT-Large2019.0932.88
GPT-KAGNETTraining Data Ratio=50%, Backbone=GPT2019.0932.33
GPT-FINETUNINGTraining Data Ratio=50%, Backbone=GPT2019.0931.28
BERT-BASE-KAGNETTraining Data Ratio=10%, Backbone=BERT-Base2019.0930.94
BERT-BASE-FINETUNINGTraining Data Ratio=10%, Backbone=BERT-Base2019.0929.78
GPT-KAGNETTraining Data Ratio=10%, Backbone=GPT2019.0926.98
GPT-FINETUNINGTraining Data Ratio=10%, Backbone=GPT2019.0926.51
Random guessTraining Data Ratio=10%2019.0920
Random guessTraining Data Ratio=50%2019.0920
Random guessTraining Data Ratio=100%2019.0920