List your company
ResearchFormerly WizWand

Commonsense Reasoning on SocialIQA

88.1AccuracyHUMAN
39.906452.418264.9377.4418Mar 24, 2021Feb 5, 2022Dec 20, 2022Nov 4, 2023Sep 17, 2024Aug 1, 2025Jun 16, 2026
Updated 4mo ago

Evaluation results

MethodLinks
HUMAN2021.0388.1——
SE-GPTModel=GPT-42024.0783.5——
UNICORN2021.0383.2——
Llama-3.3-70B-InstructShots=52026.0480.86——
Self-EXPModel=GPT-42024.0780.8——
Zero-shot-CoTModel=GPT-42024.0780.5——
DeepSeek-V3.2Access=Closed-source2026.0180.19——
UGAMIX2021.0380——
UNIFIEDQA-3B2021.0379.8——
SE-GPTModel=GPT-3.52024.0779.2——
GPT-4oAccess=Closed-source2026.0179.02——
Zero-shotModel=GPT-42024.0778.8——
Qwen3-14BShots=52026.0477.43——
GPT3.5Source Task=RACE2024.0577.2——
ROBERTA-LARGE2021.0376.7——
Qwen3.5-9BShots=52026.0476.56——
GPT3.5Source Task=ARC-Easy2024.0576.2——
GPT3.5Source Task=Zero-shot2024.0576——
GPT3.5Source Task=AG-news2024.0575.8——
GPT3.5Source Task=MNLI2024.0575.8——
Zero-shotModel=GPT-3.52024.0775.4——
GPT3.5Source Task=Commonsense-QA2024.0575.2——
Zero-shot-CoTModel=GPT-3.52024.0775.1——
Gemini3-FlashAccess=Closed-source2026.0174.87——
GPT3.5Source Task=SST22024.0574.8——
Modified Self-ICLModel=GPT-3.52024.0774.6——
Qwen3-8BShots=52026.0474.51——
GPT3.5Source Task=BoolQ2024.0574——
GPT3.5Source Task=Conll2003-NER2024.0574——
Self-ICLModel=GPT-42024.0774——
AutoP-ICLModel=GPT-3.52024.0773.5——
Self-ICL-CoTModel=GPT-42024.0773.4——
XekRung-8BShots=52026.0473.39——
GPT3.5Source Task=Conll2003-POS2024.0573.2——
SecGPT-14BShots=52026.0473.08——
GPT3.5Source Task=QQP2024.0573——
Self-EXPModel=GPT-3.52024.0771.2——
Self-ICL-CoTModel=GPT-3.52024.0769.6——
Self-ICLModel=GPT-3.52024.0769.3——
CKT-baseBackbone=T5-base, Method=CKT2023.0667.3——
CKT w/ GPT-2Backbone=GPT-2, Method=CKT2023.0666.2——
CALMBackbone=T5-base2023.0666——
T5-base + CSKG (Rule)Backbone=T5-base, Augmentation=CSKG (Rule)2023.0665.7——
T5-base + SSMBackbone=T5-base, Augmentation=SSM2023.0665.5——
KnowBERTBackbone=KnowBERT2023.0665.4——
T5-base + TIBackbone=T5-base, Augmentation=TI2023.0665.3——
ERNIE-baseBackbone=ERNIE-base2023.0665.1——
T5-baseBackbone=T5-base2023.0665.1——
T5-base + KDBackbone=T5-base, Augmentation=KD2023.0664.8——
BERT-baseBackbone=BERT-base2023.0664.3——
LLaMA-2 13BSource Task=RACE2024.0563.7——
LLaMA-2 13BSource Task=Commonsense-QA2024.0563.5——
T5-base + CSKG (TI)Backbone=T5-base, Augmentation=CSKG (TI)2023.0662.7——
LLaMA-2 13BSource Task=ARC-Easy2024.0560.8——
COMETBackbone=BART2023.0660.2——
LLaMA-2 13BSource Task=Conll2003-POS2024.0556.8——
LLaMA-2 13BSource Task=BoolQ2024.0556.5——
LLaMA-2 13BSource Task=Conll2003-NER2024.0555.9——
LLaMA-2 13BSource Task=AG-news2024.0555.8——
LLaMA-2 13BSource Task=QQP2024.0555.7——
LLaMA-2 13BSource Task=Zero-shot2024.0555.3——
OLMo 2Type=Sub-word reference2026.0255.1——
Dense FATarget Sparsity (ρ)=0, Model=1.7B2026.0654.86——
Bolmo2026.0254.7——
BLT2026.0254.6——
ConSA (head-wise, all-layers)Target Sparsity (ρ)=0.50, Granularity=head-wise, Constraint Scope=all-layers, Model=1.7B2026.0654.4——
ConSA (head-wise, single-layer)Target Sparsity (ρ)=0.50, Granularity=head-wise, Constraint Scope=single-layer, Model=1.7B2026.0653.94——
ConSA (layer-wise)Target Sparsity (ρ)=0.50, Granularity=layer-wise, Model=1.7B2026.0653.89——
OursTraining Stage=Stage 12026.0253.8——
LLaMA-2 13BSource Task=MNLI2024.0553.5——
Rule (head-wise)Target Sparsity (ρ)=0.50, Granularity=head-wise, Model=1.7B2026.0653.28——
LLaMA-2 13BSource Task=SST22024.0553.1——
OursTraining Stage=Stage 22026.0252.9——
HnetTraining Stage=2-stage2026.0252.9——
Rule (layer-wise)Target Sparsity (ρ)=0.50, Granularity=layer-wise, Model=1.7B2026.0652.3——
Qwen2.5Size=1.54B, Pre-training tokens=18T2025.1250.6——
HnetTraining Stage=1-stage2026.0250.1——
LLaMA-2 7BSource Task=RACE2024.0549.1——
Olmo2Size=1.48B, Pre-training tokens=4T2025.1248.9——
LLaMA-2 7BSource Task=Commonsense-QA2024.0548.5——
GamayunSize=1.47B, Pre-training tokens=2.6T2025.1248.2——
Qwen3Size=1.7B, Pre-training tokens=36T2025.1245.2——
EuroLMSize=1.66B, Pre-training tokens=4T2025.1244.8——
SmolLM2Size=1.7B, Pre-training tokens=11T2025.1244.8——
NTPSize=7B, Zero-shot=true2025.0844.37——
LLaMA-2 7BSource Task=ARC-Easy2024.0544.3——
MTPSize=7B, Zero-shot=true2025.0844.11——
TOPSize=7B, Zero-shot=true2025.0843.91——
DS-MTPSize=7B, Zero-shot=true2025.0843.76——
WaterSICRate=4.00, Backbone=Llama-3.2-1B, Zero-shot=true2026.0343.4——
Memory Grafting# Shots=0-shot, # Trainable Params=2.8B, # Activated (w/o token embed)=0.55B, # Trained Tokens=100B, # Experts (shared + routed, top-k)=1 + 47 (top-4)2026.0542.94——
DS-MTPSize=1.8B, Zero-shot=true2025.0842.84——
TOPSize=1.8B, Zero-shot=true2025.0842.53——
WaterSICRate=3.00, Backbone=Llama-3.2-1B, Zero-shot=true2026.0342.48——
WaterSICRate=3.50, Backbone=Llama-3.2-1B, Zero-shot=true2026.0342.17——
MTPSize=1.8B, Zero-shot=true2025.0842.12——
Huffman-GPTQRate=4.00, Backbone=Llama-3.2-1B, Zero-shot=true2026.0342.12——
MDM-Prime-v2Zero-shot=true, Number of Parameters=1.1B2026.0342.02——
Gemma3Size=1B, Pre-training tokens=2T2025.1241.8——
Vanilla Engram# Shots=0-shot, # Trainable Params=2.8B, # Activated (w/o token embed)=0.55B, # Trained Tokens=100B, # Experts (shared + routed, top-k)=1 + 48 (top-4)2026.0541.76——