Commonsense Reasoning on CommonsenseQA (CSQA) v1.0 (test)
64.11AccuracyCKT-base
Evaluation Results
| Method | Links | |
|---|---|---|
| CKT-baseBackbone=T5-base, Method=CKT2023.06 | 64.11 | |
| CALMBackbone=T5-base2023.06 | 63.32 | |
| T5-base + CSKG (Rule)Backbone=T5-base, Augmentation=CSKG (Rule)2023.06 | 63.1 | |
| T5-base + SSMBackbone=T5-base, Augmentation=SSM2023.06 | 62.37 | |
| T5-base + TIBackbone=T5-base, Augmentation=TI2023.06 | 62.05 | |
| T5-baseBackbone=T5-base, Model Size=Base2023.06 | 61.88 | |
| T5-base + KDBackbone=T5-base, Augmentation=KD2023.06 | 61.83 | |
| CKT w/ GPT-2Backbone=GPT-2, Method=CKT2023.06 | 60.39 | |
| T5-base + CSKG (TI)Backbone=T5-base, Augmentation=CSKG (TI)2023.06 | 60.22 | |
| ERNIE-baseBackbone=ERNIE-base, Model Size=Base2023.06 | 54.06 | |
| KnowBERTBackbone=KnowBERT, Model Size=Base2023.06 | 53.88 | |
| BERT-baseBackbone=BERT-base, Model Size=Base2023.06 | 53.08 | |
| COMETBackbone=BART, Model Size=Base2023.06 | 45.32 | |
| TituLLM-3b-v2.0Shots=52025.02 | 33 | |
| BLOOM-3bShots=52025.02 | 32 | |
| BLOOM-1b7Shots=52025.02 | 30 | |
| Llama-3.2-3bShots=52025.02 | 29 | |
| Gemma-2-2bShots=52025.02 | 28 | |
| TituLLM-1b-v2.0Shots=52025.02 | 28 | |
| TituLLM-3b-v2.0Shots=02025.02 | 28 | |
| BLOOM-1b1Shots=02025.02 | 27 | |
| BLOOM-1b1Shots=52025.02 | 27 | |
| BLOOM-1b7Shots=02025.02 | 27 | |
| BLOOM-3bShots=02025.02 | 27 | |
| Llama-3.2-3bShots=02025.02 | 26 | |
| Gemma-2-2bShots=02025.02 | 26 | |
| BLOOM-560mShots=02025.02 | 26 | |
| BLOOM-560mShots=52025.02 | 26 | |
| TituLLM-1b-v2.0Shots=02025.02 | 26 | |
| BongLLaMA-3.2-1bShots=52025.02 | 24 | |
| Llama-3.2-1bShots=02025.02 | 23 | |
| Llama-3.2-1bShots=52025.02 | 23 | |
| Qwen-2.5-1.5bShots=02025.02 | 23 | |
| Qwen-2.5-1.5bShots=52025.02 | 23 | |
| davinciShots=02025.02 | 22 | |
| Qwen-2.5-0.5bShots=52025.02 | 22 | |
| SmolLM2-135mShots=02025.02 | 22 | |
| SmolLM2-1.7bShots=02025.02 | 22 | |
| BongLLaMA-3.2-1bShots=02025.02 | 22 | |
| Qwen-2.5-0.5bShots=02025.02 | 21 | |
| SmolLM2-135mShots=52025.02 | 21 | |
| SmolLM2-360mShots=52025.02 | 21 | |
| SmolLM2-1.7bShots=52025.02 | 21 | |
| BongLLaMA-3.2-3bShots=02025.02 | 21 | |
| SmolLM2-360mShots=02025.02 | 20 | |
| BongLLaMA-3.2-3bShots=52025.02 | 20 |