List your company
ResearchFormerly WizWand

Commonsense Question Answering on CosmosQA

94AccuracyHUMAN
15.438435.834256.2376.6258Feb 5, 2020Feb 19, 2021Mar 7, 2022Mar 23, 2023Apr 6, 2024Apr 22, 2025May 8, 2026
Updated 6mo ago

Evaluation results

MethodLinks
HUMAN2021.0394
UNICORN2021.0391.8
QA+CCoverage=20%, Base model=RoBERTa-RACE2022.0991.12
PACSBackbone=Llama 3.3 70B2026.0591
T5-11BParameters=11B2021.0390.3
SC-20Backbone=Llama 3.3 70B2026.0590
ARGOSBackbone=Llama 3.3 70B2026.0590
QACoverage=20%, Base model=RoBERTa-RACE2022.0988.61
COTBackbone=Llama 3.3 70B2026.0588
If-BeamBackbone=Llama 3.3 70B2026.0588
ALBERT-XXLARGEModel Size=XXLarge2021.0385.4
LoTBackbone=Llama 3.3 70B2026.0585
ROBERTA-LARGEModel Size=Large2021.0383.5
CCoverage=20%, Base model=RoBERTa-RACE2022.0983.25
ORACLEBackbone=Llama-3.1-8B-Instruct, Setting=zero-shot2026.0382.4
DRAGON2022.1082.3
ORACLEBackbone=Qwen-2.5-7B-instruct, Setting=zero-shot2026.0382.1
PACSBackbone=Llama 3-Instruct 8B2026.0582
ARGOSBackbone=Llama 3-Instruct 8B2026.0582
If-BeamBackbone=Llama 3-Instruct 8B2026.0582
K-ADAPTER (F+L)Adapter type=facAdapter + linAdapter2020.0281.83
QA+CCoverage=50%, Base model=RoBERTa-RACE2022.0981.7
self-rewardingBackbone=Llama-3.1-8B-Instruct, Setting=zero-shot2026.0381.5
RoBERTa + multitaskmultitask=true2020.0281.19
K-ADAPTER (F)Adapter type=facAdapter2020.0280.93
K-ADAPTER (L)Adapter type=linAdapter2020.0280.76
QAGNN2022.1080.7
CCoverage=50%, Base model=RoBERTa-RACE2022.0980.63
GreaseLM2022.1080.6
RoBERTa2020.0280.59
RoBERTa2022.1080.5
RFTBackbone=Llama-3.1-8B-Instruct, Setting=zero-shot2026.0380.4
QA+E+CCoverage=50%, Base model=RoBERTa-RACE2022.0980.29
SC-20Backbone=Llama 3-Instruct 8B2026.0580
CoTBackbone=Llama-3.1-8B-Instruct, Setting=four-shot2026.0379.5
ORACLEBackbone=Mistral-7B-Instruct-v0.3, Setting=zero-shot2026.0378.5
self-rewardingBackbone=Qwen-2.5-7B-instruct, Setting=zero-shot2026.0378.1
QA+E+CCoverage=20%, Base model=RoBERTa-RACE2022.0977.55
CoTBackbone=Qwen-2.5-7B-instruct, Setting=four-shot2026.0377.1
QA+ECoverage=50%, Base model=RoBERTa-RACE2022.0976.94
QA+ECoverage=20%, Base model=RoBERTa-RACE2022.0976.88
QACoverage=50%, Base model=RoBERTa-RACE2022.0976.47
RFTBackbone=Qwen-2.5-7B-instruct, Setting=zero-shot2026.0376
LoTBackbone=Llama 3-Instruct 8B2026.0576
CoTBackbone=Qwen-2.5-7B-instruct, Setting=zero-shot2026.0375.9
E+CCoverage=50%, Base model=RoBERTa-RACE2022.0975.8
COTBackbone=Llama 3-Instruct 8B2026.0575
ToT-SFTBackbone=Llama-3.1-8B-Instruct, Setting=zero-shot2026.0374.7
CoTBackbone=Llama-3.1-8B-Instruct, Setting=zero-shot2026.0374.2
RFTBackbone=Mistral-7B-Instruct-v0.3, Setting=zero-shot2026.0374
ToT-SFTBackbone=Qwen-2.5-7B-instruct, Setting=zero-shot2026.0373.9
ToT-SFTBackbone=Mistral-7B-Instruct-v0.3, Setting=zero-shot2026.0372.4
self-rewardingBackbone=Mistral-7B-Instruct-v0.3, Setting=zero-shot2026.0371.7
CoTBackbone=Mistral-7B-Instruct-v0.3, Setting=four-shot2026.0371.1
ECoverage=50%, Base model=RoBERTa-RACE2022.0970.64
CoTBackbone=Mistral-7B-Instruct-v0.3, Setting=zero-shot2026.0370.6
E+CCoverage=20%, Base model=RoBERTa-RACE2022.0969.18
BERT-FT RACE+SWAGpre-training datasets=RACE + SWAG2020.0268.7
ECoverage=20%, Base model=RoBERTa-RACE2022.0968.34
QA+Cmodel=RoBERTa-RACE, contradiction=true2022.0964.82
QA+E+Cmodel=RoBERTa-RACE, entailment=true, contradiction=true2022.0964.72
QA+Emodel=RoBERTa-RACE, entailment=true2022.0964.32
QAmodel=RoBERTa-RACE2022.0964.22
LLM-TreeBackbone=Llama 3-Instruct 8B2026.0551
LLM-TreeBackbone=Llama 3.3 70B2026.0551
OLTQAFull Config=True2023.0546.16
SciQ-smallvariant=small2022.0925.46
SciQ-basevariant=base2022.0918.46