Question Answering on CommonsenseQA IH (dev)
82.7AccuracyKnowGPT
Evaluation Results
| Method | Links | |
|---|---|---|
| KnowGPTCategory=Ours2023.12 | 82.7 | |
| Ours2023.05 | 79.8 | |
| GSC2023.05 | 79.11 | |
| MindmapCategory=LLM + KG Prompting2023.12 | 78.9 | |
| GreaseLM2023.05 | 78.5 | |
| GreaseLMCategory=KG-enhanced LM2023.12 | 78.5 | |
| GrapeQACategory=KG-enhanced LM2023.12 | 78.2 | |
| JointLKCategory=KG-enhanced LM2023.12 | 77.7 | |
| GPT-4Category=LLM + Zero-shot2023.12 | 77.6 | |
| HamQACategory=KG-enhanced LM2023.12 | 76.9 | |
| QA-GNN2023.05 | 76.54 | |
| QA-GNNCategory=KG-enhanced LM2023.12 | 76.5 | |
| CoKCategory=LLM + KG Prompting2023.12 | 75.9 | |
| RoGCategory=LLM + KG Prompting2023.12 | 75 | |
| RN2023.05 | 74.57 | |
| MHGRNCategory=KG-enhanced LM2023.12 | 74.5 | |
| Llama3Category=LLM + Zero-shot, Parameters=8b2023.12 | 74.5 | |
| MHGRN2023.05 | 74.45 | |
| GPT-3.5Category=LLM + Zero-shot, Implementation=gpt-3.5-turbo2023.12 | 73.5 | |
| KagNet2023.05 | 73.47 | |
| RoBerta-largeCategory=LM + Fine tuning2023.12 | 73.1 | |
| RoBERTa-largeKnowledge Graph=without2023.05 | 73.07 | |
| RGCN2023.05 | 72.69 | |
| GconAttn2023.05 | 72.61 | |
| BERT-LARGE-KAGNETTraining Data Ratio=100%, Backbone=BERT-Large2019.09 | 62.35 | |
| Bert-largeCategory=LM + Fine tuning2023.12 | 61.1 | |
| BERT-LARGE-FINETUNINGTraining Data Ratio=100%, Backbone=BERT-Large2019.09 | 60.61 | |
| BERT-LARGE-KAGNETTraining Data Ratio=50%, Backbone=BERT-Large2019.09 | 58.73 | |
| Bert-baseCategory=LM + Fine tuning2023.12 | 57.3 | |
| Llama2Category=LLM + Zero-shot, Parameters=7b2023.12 | 56.4 | |
| BERT-BASE-KAGNETTraining Data Ratio=100%, Backbone=BERT-Base2019.09 | 55.57 | |
| BERT-LARGE-FINETUNINGTraining Data Ratio=50%, Backbone=BERT-Large2019.09 | 55.45 | |
| GPT-3Category=LLM + Zero-shot, Implementation=text-davinci-0022023.12 | 53.9 | |
| BERT-BASE-FINETUNINGTraining Data Ratio=100%, Backbone=BERT-Base2019.09 | 53.48 | |
| BaichuanCategory=LLM + Zero-shot, Parameters=7B2023.12 | 49.1 | |
| GPT-KAGNETTraining Data Ratio=100%, Backbone=GPT2019.09 | 48.95 | |
| InternLMCategory=LLM + Zero-shot2023.12 | 47.7 | |
| GPT-FINETUNINGTraining Data Ratio=100%, Backbone=GPT2019.09 | 47.35 | |
| ChatGLMCategory=LLM + Zero-shot2023.12 | 47.3 | |
| ChatGLM2Category=LLM + Zero-shot2023.12 | 44 | |
| BERT-BASE-KAGNETTraining Data Ratio=50%, Backbone=BERT-Base2019.09 | 40.32 | |
| BERT-BASE-FINETUNINGTraining Data Ratio=50%, Backbone=BERT-Base2019.09 | 38.66 | |
| BERT-LARGE-KAGNETTraining Data Ratio=10%, Backbone=BERT-Large2019.09 | 36.82 | |
| BERT-LARGE-FINETUNINGTraining Data Ratio=10%, Backbone=BERT-Large2019.09 | 35.71 | |
| GPT-KAGNETTraining Data Ratio=50%, Backbone=GPT2019.09 | 33.72 | |
| GPT-FINETUNINGTraining Data Ratio=50%, Backbone=GPT2019.09 | 32.46 | |
| BERT-BASE-KAGNETTraining Data Ratio=10%, Backbone=BERT-Base2019.09 | 31.05 | |
| BERT-BASE-FINETUNINGTraining Data Ratio=10%, Backbone=BERT-Base2019.09 | 30.11 | |
| GPT-KAGNETTraining Data Ratio=10%, Backbone=GPT2019.09 | 28.13 | |
| GPT-FINETUNINGTraining Data Ratio=10%, Backbone=GPT2019.09 | 27.55 | |
| Random guessTraining Data Ratio=10%2019.09 | 20 | |
| Random guessTraining Data Ratio=50%2019.09 | 20 | |
| Random guessTraining Data Ratio=100%2019.09 | 20 |