Commonsense Reasoning on CSQA (dev)
85.42AccuracyRandom sentences
Evaluation results
| Method | Links | |
|---|---|---|
| Random sentencesInference Model=UQA-11b-ft2021.10 | 85.42 | |
| Context sentencesInference Model=UQA-11b-ft2021.10 | 85.34 | |
| Generated Knowledge PromptingInference Model=UQA-11b-ft2021.10 | 85.34 | |
| Vanilla baselineInference Model=UQA-11b-ft2021.10 | 85.18 | |
| AnswersInference Model=UQA-11b-ft2021.10 | 84.93 | |
| UQA-11b-ft (prev. SOTA)Note=Value reported from test set2021.10 | 79.1 | |
| CKTModel Scale=large2023.06 | 72.15 | |
| Few-shot GPT-3 Infer.2021.10 | 71.58 | |
| CALMModel Scale=large2023.06 | 71.31 | |
| T5Model Scale=large2023.06 | 69.81 | |
| AnswersInference Model=T5-11b2021.10 | 51.84 | |
| Generated Knowledge PromptingInference Model=T5-11b2021.10 | 47.26 | |
| Template-based knowledgeInference Model=T5-11b2021.10 | 45.37 | |
| Context sentencesInference Model=T5-11b2021.10 | 42.51 | |
| Vanilla baselineInference Model=T5-11b2021.10 | 39.89 | |
| Random sentencesInference Model=T5-11b2021.10 | 21.79 |