Commonsense Reasoning on SocialIQA
88.1AccuracyHUMAN
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| HUMAN2021.03 | 88.1 | — | — | |
| SE-GPTModel=GPT-42024.07 | 83.5 | — | — | |
| UNICORN2021.03 | 83.2 | — | — | |
| Llama-3.3-70B-InstructShots=52026.04 | 80.86 | — | — | |
| Self-EXPModel=GPT-42024.07 | 80.8 | — | — | |
| Zero-shot-CoTModel=GPT-42024.07 | 80.5 | — | — | |
| DeepSeek-V3.2Access=Closed-source2026.01 | 80.19 | — | — | |
| UGAMIX2021.03 | 80 | — | — | |
| UNIFIEDQA-3B2021.03 | 79.8 | — | — | |
| SE-GPTModel=GPT-3.52024.07 | 79.2 | — | — | |
| GPT-4oAccess=Closed-source2026.01 | 79.02 | — | — | |
| Zero-shotModel=GPT-42024.07 | 78.8 | — | — | |
| Qwen3-14BShots=52026.04 | 77.43 | — | — | |
| GPT3.5Source Task=RACE2024.05 | 77.2 | — | — | |
| ROBERTA-LARGE2021.03 | 76.7 | — | — | |
| Qwen3.5-9BShots=52026.04 | 76.56 | — | — | |
| GPT3.5Source Task=ARC-Easy2024.05 | 76.2 | — | — | |
| GPT3.5Source Task=Zero-shot2024.05 | 76 | — | — | |
| GPT3.5Source Task=AG-news2024.05 | 75.8 | — | — | |
| GPT3.5Source Task=MNLI2024.05 | 75.8 | — | — | |
| Zero-shotModel=GPT-3.52024.07 | 75.4 | — | — | |
| GPT3.5Source Task=Commonsense-QA2024.05 | 75.2 | — | — | |
| Zero-shot-CoTModel=GPT-3.52024.07 | 75.1 | — | — | |
| Gemini3-FlashAccess=Closed-source2026.01 | 74.87 | — | — | |
| GPT3.5Source Task=SST22024.05 | 74.8 | — | — | |
| Modified Self-ICLModel=GPT-3.52024.07 | 74.6 | — | — | |
| Qwen3-8BShots=52026.04 | 74.51 | — | — | |
| GPT3.5Source Task=BoolQ2024.05 | 74 | — | — | |
| GPT3.5Source Task=Conll2003-NER2024.05 | 74 | — | — | |
| Self-ICLModel=GPT-42024.07 | 74 | — | — | |
| AutoP-ICLModel=GPT-3.52024.07 | 73.5 | — | — | |
| Self-ICL-CoTModel=GPT-42024.07 | 73.4 | — | — | |
| XekRung-8BShots=52026.04 | 73.39 | — | — | |
| GPT3.5Source Task=Conll2003-POS2024.05 | 73.2 | — | — | |
| SecGPT-14BShots=52026.04 | 73.08 | — | — | |
| GPT3.5Source Task=QQP2024.05 | 73 | — | — | |
| Self-EXPModel=GPT-3.52024.07 | 71.2 | — | — | |
| Self-ICL-CoTModel=GPT-3.52024.07 | 69.6 | — | — | |
| Self-ICLModel=GPT-3.52024.07 | 69.3 | — | — | |
| CKT-baseBackbone=T5-base, Method=CKT2023.06 | 67.3 | — | — | |
| CKT w/ GPT-2Backbone=GPT-2, Method=CKT2023.06 | 66.2 | — | — | |
| CALMBackbone=T5-base2023.06 | 66 | — | — | |
| T5-base + CSKG (Rule)Backbone=T5-base, Augmentation=CSKG (Rule)2023.06 | 65.7 | — | — | |
| T5-base + SSMBackbone=T5-base, Augmentation=SSM2023.06 | 65.5 | — | — | |
| KnowBERTBackbone=KnowBERT2023.06 | 65.4 | — | — | |
| T5-base + TIBackbone=T5-base, Augmentation=TI2023.06 | 65.3 | — | — | |
| ERNIE-baseBackbone=ERNIE-base2023.06 | 65.1 | — | — | |
| T5-baseBackbone=T5-base2023.06 | 65.1 | — | — | |
| T5-base + KDBackbone=T5-base, Augmentation=KD2023.06 | 64.8 | — | — | |
| BERT-baseBackbone=BERT-base2023.06 | 64.3 | — | — | |
| LLaMA-2 13BSource Task=RACE2024.05 | 63.7 | — | — | |
| LLaMA-2 13BSource Task=Commonsense-QA2024.05 | 63.5 | — | — | |
| T5-base + CSKG (TI)Backbone=T5-base, Augmentation=CSKG (TI)2023.06 | 62.7 | — | — | |
| LLaMA-2 13BSource Task=ARC-Easy2024.05 | 60.8 | — | — | |
| COMETBackbone=BART2023.06 | 60.2 | — | — | |
| LLaMA-2 13BSource Task=Conll2003-POS2024.05 | 56.8 | — | — | |
| LLaMA-2 13BSource Task=BoolQ2024.05 | 56.5 | — | — | |
| LLaMA-2 13BSource Task=Conll2003-NER2024.05 | 55.9 | — | — | |
| LLaMA-2 13BSource Task=AG-news2024.05 | 55.8 | — | — | |
| LLaMA-2 13BSource Task=QQP2024.05 | 55.7 | — | — | |
| LLaMA-2 13BSource Task=Zero-shot2024.05 | 55.3 | — | — | |
| OLMo 2Type=Sub-word reference2026.02 | 55.1 | — | — | |
| Dense FATarget Sparsity (ρ)=0, Model=1.7B2026.06 | 54.86 | — | — | |
| Bolmo2026.02 | 54.7 | — | — | |
| BLT2026.02 | 54.6 | — | — | |
| ConSA (head-wise, all-layers)Target Sparsity (ρ)=0.50, Granularity=head-wise, Constraint Scope=all-layers, Model=1.7B2026.06 | 54.4 | — | — | |
| ConSA (head-wise, single-layer)Target Sparsity (ρ)=0.50, Granularity=head-wise, Constraint Scope=single-layer, Model=1.7B2026.06 | 53.94 | — | — | |
| ConSA (layer-wise)Target Sparsity (ρ)=0.50, Granularity=layer-wise, Model=1.7B2026.06 | 53.89 | — | — | |
| OursTraining Stage=Stage 12026.02 | 53.8 | — | — | |
| LLaMA-2 13BSource Task=MNLI2024.05 | 53.5 | — | — | |
| Rule (head-wise)Target Sparsity (ρ)=0.50, Granularity=head-wise, Model=1.7B2026.06 | 53.28 | — | — | |
| LLaMA-2 13BSource Task=SST22024.05 | 53.1 | — | — | |
| OursTraining Stage=Stage 22026.02 | 52.9 | — | — | |
| HnetTraining Stage=2-stage2026.02 | 52.9 | — | — | |
| Rule (layer-wise)Target Sparsity (ρ)=0.50, Granularity=layer-wise, Model=1.7B2026.06 | 52.3 | — | — | |
| Qwen2.5Size=1.54B, Pre-training tokens=18T2025.12 | 50.6 | — | — | |
| HnetTraining Stage=1-stage2026.02 | 50.1 | — | — | |
| LLaMA-2 7BSource Task=RACE2024.05 | 49.1 | — | — | |
| Olmo2Size=1.48B, Pre-training tokens=4T2025.12 | 48.9 | — | — | |
| LLaMA-2 7BSource Task=Commonsense-QA2024.05 | 48.5 | — | — | |
| GamayunSize=1.47B, Pre-training tokens=2.6T2025.12 | 48.2 | — | — | |
| Qwen3Size=1.7B, Pre-training tokens=36T2025.12 | 45.2 | — | — | |
| EuroLMSize=1.66B, Pre-training tokens=4T2025.12 | 44.8 | — | — | |
| SmolLM2Size=1.7B, Pre-training tokens=11T2025.12 | 44.8 | — | — | |
| NTPSize=7B, Zero-shot=true2025.08 | 44.37 | — | — | |
| LLaMA-2 7BSource Task=ARC-Easy2024.05 | 44.3 | — | — | |
| MTPSize=7B, Zero-shot=true2025.08 | 44.11 | — | — | |
| TOPSize=7B, Zero-shot=true2025.08 | 43.91 | — | — | |
| DS-MTPSize=7B, Zero-shot=true2025.08 | 43.76 | — | — | |
| WaterSICRate=4.00, Backbone=Llama-3.2-1B, Zero-shot=true2026.03 | 43.4 | — | — | |
| Memory Grafting# Shots=0-shot, # Trainable Params=2.8B, # Activated (w/o token embed)=0.55B, # Trained Tokens=100B, # Experts (shared + routed, top-k)=1 + 47 (top-4)2026.05 | 42.94 | — | — | |
| DS-MTPSize=1.8B, Zero-shot=true2025.08 | 42.84 | — | — | |
| TOPSize=1.8B, Zero-shot=true2025.08 | 42.53 | — | — | |
| WaterSICRate=3.00, Backbone=Llama-3.2-1B, Zero-shot=true2026.03 | 42.48 | — | — | |
| WaterSICRate=3.50, Backbone=Llama-3.2-1B, Zero-shot=true2026.03 | 42.17 | — | — | |
| MTPSize=1.8B, Zero-shot=true2025.08 | 42.12 | — | — | |
| Huffman-GPTQRate=4.00, Backbone=Llama-3.2-1B, Zero-shot=true2026.03 | 42.12 | — | — | |
| MDM-Prime-v2Zero-shot=true, Number of Parameters=1.1B2026.03 | 42.02 | — | — | |
| Gemma3Size=1B, Pre-training tokens=2T2025.12 | 41.8 | — | — | |
| Vanilla Engram# Shots=0-shot, # Trainable Params=2.8B, # Activated (w/o token embed)=0.55B, # Trained Tokens=100B, # Experts (shared + routed, top-k)=1 + 48 (top-4)2026.05 | 41.76 | — | — |