List your company
ResearchFormerly WizWand

Common Sense Reasoning on HSWAG

0.9751AccuracyIn-Squeeze
0.2407560.4314030.622050.812697May 23, 2024Sep 4, 2024Dec 18, 2024Apr 2, 2025Jul 16, 2025Oct 29, 2025Feb 11, 2026
Updated 8mo ago

Evaluation results

MethodLinks
In-Squeezerank=128 to 1, schedule=Min steps2026.020.9751
In-Squeezerank=128 to 1, schedule=Standard2026.020.9749
Cont-Squeezerank=128 to 1, additional_steps=2002026.020.9742
Direct Fine-tuningrank=1, additional_steps=02026.020.9719
Direct Fine-tuningrank=1, additional_steps=2002026.020.9719
Cont-Squeezerank=128 to 1, additional_steps=7002026.020.9716
Direct Fine-tuningrank=1, additional_steps=7002026.020.9709
Cont-Squeezerank=128 to 1, additional_steps=02026.020.9679
In-Squeezereduction=128 ... -> 1, strategy=Min steps2026.020.9058
In-Squeezereduction=128 ... -> 1, strategy=Standard2026.020.8976
Direct Fine-tuningrank=1, training_steps=+0 steps2026.020.8914
Cont-Squeezereduction=128 -> 1, training_steps=+700 steps2026.020.8889
Cont-Squeezereduction=128 -> 1, training_steps=+200 steps2026.020.8841
Direct Fine-tuningrank=1, training_steps=+700 steps2026.020.8799
Direct Fine-tuningrank=1, training_steps=+200 steps2026.020.8711
Qwen3-4BParams=4B2025.120.7946
Llama-3.1Size=8B, Type=AR2026.010.76
Qwen2.5-3BParams=3B2025.120.7385
llama-3.2-3BParams=3B2025.120.7106
SmolLM3-3BParams=3B2025.120.692
GenKnowSubBase Model=Phi-2, Setting=Avg, Evaluation Protocol=zero-shot2025.050.6796
Phi-2Base Model=Phi-2, Evaluation Protocol=zero-shot2025.050.6675
GenKnowSubBase Model=Phi-2, Setting=En, Evaluation Protocol=zero-shot2025.050.6675
ArrowBase Model=Phi-2, Evaluation Protocol=zero-shot2025.050.6641
gemma2-2BParams=2B2025.120.662
Cont-Squeezereduction=128 -> 1, training_steps=+0 steps2026.020.6555
SmolLM2-1.7BParams=1.7B2025.120.653
Qwen2.5-1.5BParams=1.5B2025.120.6418
OLMo-2-0425-1BParams=1B2025.120.6161
YuLan-Mini-2.4BParams=2.4B2025.120.612
Qwen2-1.5BParams=1.5B2025.120.6077
Qwen3-1.7BParams=1.7B2025.120.6076
llama-3.2-1BParams=1B2025.120.602
DiffuLlamaSize=7B, Type=DD2026.010.587
A3Size=8B, Type=A32026.010.584
PCMind-2.1-Kaiyuan-2BParams=2B2025.120.5813
Zero-shotmode=0-S2026.020.508
A3Size=3B, Type=A32026.010.496
Qwen3-0.6BParams=0.6B2025.120.4851
EMR-MERGINGValidation=false2024.050.447
Individual2024.050.444
Ties-MergingValidation=true2024.050.428
Ties-MergingValidation=false2024.050.426
Traditional MTL2024.050.423
Fisher MergingValidation=true2024.050.422
Task ArithmeticValidation=true2024.050.415
A3Size=1B, Type=A32026.010.402
RegMeanValidation=true2024.050.401
Weight AveragingValidation=false2024.050.401
PlaidSize=1B, Type=CD2026.010.393
Task ArithmeticValidation=false2024.050.314
DreamSize=7B, Type=DD2026.010.269