Question Answering on CommonsenseQA (test)
83.3AccuracyDEKCOR
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DEKCORModel Configuration=Ensemble2020.12 | 83.3 | — | |
| Linear ProbeTrain Dataset=ComQA2026.03 | 81.98 | — | |
| DEKCORModel Configuration=Single2020.12 | 80.7 | — | |
| ALBERT+KCRModel Configuration=Single2020.12 | 79.5 | — | |
| UnifiedQAModel Configuration=Single2020.12 | 79.1 | — | |
| UnifiedQAnumber of parameters=11B2021.10 | 79.1 | — | |
| UnifiedQANumber of Parameters=11B2021.04 | 79.1 | — | |
| Linear ProbeTrain Dataset=ARC-E2026.03 | 78.55 | — | |
| ALBERT+KRDModel Configuration=Single2020.12 | 78.4 | — | |
| ALBERT+PG-FullModel Configuration=Ensemble2020.12 | 78.2 | — | |
| T5Model Configuration=Single2020.12 | 78.1 | — | |
| TaTTrain Dataset=ComQA2026.03 | 77.56 | — | |
| Linear ProbeTrain Dataset=CosQA2026.03 | 77.34 | — | |
| ALBERTModel Configuration=Ensemble2020.12 | 76.5 | — | |
| RoBERTa+MHGRNModel Configuration=Ensemble2020.12 | 76.5 | — | |
| ALBERTensemble=true2021.10 | 76.5 | — | |
| AlbertBackbone=Albert, Ensemble=true2021.04 | 76.5 | — | |
| ROBERTa + GSC2021.10 | 76.2 | — | |
| ROBERTA + QA-GNN2021.10 | 76.1 | — | |
| ROBERTa + QA-GNNBackbone=ROBERTa-large, Number of Parameters=~360M2021.04 | 76.1 | — | |
| ALBERT+PG-FullModel Configuration=Single2020.12 | 75.6 | — | |
| ALBERT + PG2021.10 | 75.6 | — | |
| Albert+PGBackbone=Albert2021.04 | 75.6 | — | |
| RoBERTa+MHGRNModel Configuration=Single2020.12 | 75.4 | — | |
| ROBERTA + MHGRN2021.10 | 75.4 | — | |
| ROBERTa+MHGRNBackbone=ROBERTa2021.04 | 75.4 | — | |
| XLNet+GraphReasonModel Configuration=Single2020.12 | 75.3 | — | |
| XLNet + GraphReason2021.10 | 75.3 | — | |
| XLNet+GraphReasonBackbone=XLNet2021.04 | 75.3 | — | |
| TaTTrain Dataset=OpenQA2026.03 | 75.02 | — | |
| Linear ProbeTrain Dataset=ARC-C2026.03 | 74.59 | — | |
| RoBERTa+KEDGNModel Configuration=Ensemble2020.12 | 74.4 | — | |
| ROBERTa + KEDGNensemble=true2021.10 | 74.4 | — | |
| ROBERTa+KEDGNBackbone=ROBERTa, Ensemble=true2021.04 | 74.4 | — | |
| XLNet+DREAMModel Configuration=Ensemble2020.12 | 73.3 | — | |
| RoBERTa+KEModel Configuration=Single2020.12 | 73.3 | — | |
| ROBERTA + KEensemble=true2021.10 | 73.3 | — | |
| ROBERTa+KEBackbone=ROBERTa, Ensemble=true2021.04 | 73.3 | — | |
| RoBERTa+HyKASModel Configuration=Single2020.12 | 73.2 | — | |
| ROBERTa + HyKAS2021.10 | 73.2 | — | |
| ROBERTa+HyKASBackbone=ROBERTa2021.04 | 73.2 | — | |
| ROBERTa + FreeLBensemble=true2021.10 | 73.1 | — | |
| ROBERTa+FreeLBBackbone=ROBERTa, Ensemble=true2021.04 | 73.1 | — | |
| Linear ProbeTrain Dataset=OpenQA2026.03 | 73.01 | — | |
| RoBERTaModel Configuration=Ensemble2020.12 | 72.5 | — | |
| TaTTrain Dataset=ARC-C2026.03 | 72.48 | — | |
| Linear ProbeTrain Dataset=SiQA2026.03 | 72.16 | — | |
| RoBERTaModel Configuration=Single2020.12 | 72.1 | — | |
| ROBERTa2021.10 | 72.1 | — | |
| ROBERTaBackbone=ROBERTa2021.04 | 72.1 | — | |
| Few-shot AccuracyMode=Few-shot2026.03 | 71.1 | — | |
| TaTTrain Dataset=ARC-E2026.03 | 71.09 | — | |
| TaTTrain Dataset=CosQA2026.03 | 64.05 | — | |
| Linear ProbeTrain Dataset=Hellaswag2026.03 | 63.34 | — | |
| TaTTrain Dataset=Hellaswag2026.03 | 62.82 | — | |
| BERT+OMCSModel Configuration=Single2020.12 | 62.5 | — | |
| Zero-shot AccuracyMode=Zero-shot2026.03 | 62.4 | — | |
| TaTTrain Dataset=SiQA2026.03 | 55.69 | — | |
| Linear ProbeTrain Dataset=BoolQ2026.03 | 53.22 | — | |
| TaTTrain Dataset=BoolQ2026.03 | 47.5 | — | |
| LLaMA-3.1-8Balpha=0.05, pairing_context=Block 1 (vs 3B)2025.12 | — | 1,579 | |
| LLaMA-3.1-8Balpha=0.1, pairing_context=Block 1 (vs 3B)2025.12 | — | 2,357 | |
| LLaMA-3.1-8Balpha=0.15, pairing_context=Block 1 (vs 3B)2025.12 | — | 2,890 | |
| LLaMA-3.1-8Balpha=0.2, pairing_context=Block 1 (vs 3B)2025.12 | — | 3,238 | |
| LLaMA-3.1-8Balpha=0.25, pairing_context=Block 1 (vs 3B)2025.12 | — | 3,549 | |
| LLaMA-3.1-8Balpha=0.3, pairing_context=Block 1 (vs 3B)2025.12 | — | 3,643 | |
| LLaMA-3.1-8Balpha=0.05, pairing_context=Block 2 (vs 7B)2025.12 | — | 1,577 | |
| LLaMA-3.1-8Balpha=0.1, pairing_context=Block 2 (vs 7B)2025.12 | — | 2,360 | |
| LLaMA-3.1-8Balpha=0.15, pairing_context=Block 2 (vs 7B)2025.12 | — | 2,892 | |
| LLaMA-3.1-8Balpha=0.2, pairing_context=Block 2 (vs 7B)2025.12 | — | 3,237 | |
| LLaMA-3.1-8Balpha=0.25, pairing_context=Block 2 (vs 7B)2025.12 | — | 3,546 | |
| LLaMA-3.1-8Balpha=0.3, pairing_context=Block 2 (vs 7B)2025.12 | — | 3,629 | |
| Qwen2.5-3Balpha=0.05, system_type=Single-model selective prediction2025.12 | — | 965 | |
| Qwen2.5-3Balpha=0.1, system_type=Single-model selective prediction2025.12 | — | 2,569 | |
| Qwen2.5-3Balpha=0.15, system_type=Single-model selective prediction2025.12 | — | 3,174 | |
| Qwen2.5-3Balpha=0.2, system_type=Single-model selective prediction2025.12 | — | 3,540 | |
| Qwen2.5-3Balpha=0.25, system_type=Single-model selective prediction2025.12 | — | 3,797 | |
| Qwen2.5-3Balpha=0.3, system_type=Single-model selective prediction2025.12 | — | 3,797 | |
| Qwen2.5-3B & LLaMA-3.1-8Balpha=0.05, system_type=LEC-Routing2025.12 | — | 1,610 | |
| Qwen2.5-3B & LLaMA-3.1-8Balpha=0.1, system_type=LEC-Routing2025.12 | — | 2,663 | |
| Qwen2.5-3B & LLaMA-3.1-8Balpha=0.15, system_type=LEC-Routing2025.12 | — | 3,293 | |
| Qwen2.5-3B & LLaMA-3.1-8Balpha=0.2, system_type=LEC-Routing2025.12 | — | 3,686 | |
| Qwen2.5-3B & LLaMA-3.1-8Balpha=0.25, system_type=LEC-Routing2025.12 | — | 3,836 | |
| Qwen2.5-3B & LLaMA-3.1-8Balpha=0.3, system_type=LEC-Routing2025.12 | — | 3,836 | |
| Qwen2.5-7Balpha=0.05, system_type=Single-model selective prediction2025.12 | — | 2,392 | |
| Qwen2.5-7Balpha=0.1, system_type=Single-model selective prediction2025.12 | — | 3,078 | |
| Qwen2.5-7Balpha=0.15, system_type=Single-model selective prediction2025.12 | — | 3,523 | |
| Qwen2.5-7Balpha=0.2, system_type=Single-model selective prediction2025.12 | — | 3,858 | |
| Qwen2.5-7Balpha=0.25, system_type=Single-model selective prediction2025.12 | — | 3,924 | |
| Qwen2.5-7Balpha=0.3, system_type=Single-model selective prediction2025.12 | — | 3,924 | |
| Qwen2.5-7B & LLaMA-3.1-8Balpha=0.05, system_type=LEC-Routing2025.12 | — | 2,413 | |
| Qwen2.5-7B & LLaMA-3.1-8Balpha=0.1, system_type=LEC-Routing2025.12 | — | 3,144 | |
| Qwen2.5-7B & LLaMA-3.1-8Balpha=0.15, system_type=LEC-Routing2025.12 | — | 3,615 | |
| Qwen2.5-7B & LLaMA-3.1-8Balpha=0.2, system_type=LEC-Routing2025.12 | — | 3,896 | |
| Qwen2.5-7B & LLaMA-3.1-8Balpha=0.25, system_type=LEC-Routing2025.12 | — | 3,928 | |
| Qwen2.5-7B & LLaMA-3.1-8Balpha=0.3, system_type=LEC-Routing2025.12 | — | 3,928 |