Common Sense Reasoning on HSWAG
0.9751AccuracyIn-Squeeze
Evaluation Results
| Method | Links | |
|---|---|---|
| In-Squeezerank=128 to 1, schedule=Min steps2026.02 | 0.9751 | |
| In-Squeezerank=128 to 1, schedule=Standard2026.02 | 0.9749 | |
| Cont-Squeezerank=128 to 1, additional_steps=2002026.02 | 0.9742 | |
| Direct Fine-tuningrank=1, additional_steps=02026.02 | 0.9719 | |
| Direct Fine-tuningrank=1, additional_steps=2002026.02 | 0.9719 | |
| Cont-Squeezerank=128 to 1, additional_steps=7002026.02 | 0.9716 | |
| Direct Fine-tuningrank=1, additional_steps=7002026.02 | 0.9709 | |
| Cont-Squeezerank=128 to 1, additional_steps=02026.02 | 0.9679 | |
| In-Squeezereduction=128 ... -> 1, strategy=Min steps2026.02 | 0.9058 | |
| In-Squeezereduction=128 ... -> 1, strategy=Standard2026.02 | 0.8976 | |
| Direct Fine-tuningrank=1, training_steps=+0 steps2026.02 | 0.8914 | |
| Cont-Squeezereduction=128 -> 1, training_steps=+700 steps2026.02 | 0.8889 | |
| Cont-Squeezereduction=128 -> 1, training_steps=+200 steps2026.02 | 0.8841 | |
| Direct Fine-tuningrank=1, training_steps=+700 steps2026.02 | 0.8799 | |
| Direct Fine-tuningrank=1, training_steps=+200 steps2026.02 | 0.8711 | |
| Qwen3-4BParams=4B2025.12 | 0.7946 | |
| Llama-3.1Size=8B, Type=AR2026.01 | 0.76 | |
| Qwen2.5-3BParams=3B2025.12 | 0.7385 | |
| llama-3.2-3BParams=3B2025.12 | 0.7106 | |
| SmolLM3-3BParams=3B2025.12 | 0.692 | |
| GenKnowSubBase Model=Phi-2, Setting=Avg, Evaluation Protocol=zero-shot2025.05 | 0.6796 | |
| Phi-2Base Model=Phi-2, Evaluation Protocol=zero-shot2025.05 | 0.6675 | |
| GenKnowSubBase Model=Phi-2, Setting=En, Evaluation Protocol=zero-shot2025.05 | 0.6675 | |
| ArrowBase Model=Phi-2, Evaluation Protocol=zero-shot2025.05 | 0.6641 | |
| gemma2-2BParams=2B2025.12 | 0.662 | |
| Cont-Squeezereduction=128 -> 1, training_steps=+0 steps2026.02 | 0.6555 | |
| SmolLM2-1.7BParams=1.7B2025.12 | 0.653 | |
| Qwen2.5-1.5BParams=1.5B2025.12 | 0.6418 | |
| OLMo-2-0425-1BParams=1B2025.12 | 0.6161 | |
| YuLan-Mini-2.4BParams=2.4B2025.12 | 0.612 | |
| Qwen2-1.5BParams=1.5B2025.12 | 0.6077 | |
| Qwen3-1.7BParams=1.7B2025.12 | 0.6076 | |
| llama-3.2-1BParams=1B2025.12 | 0.602 | |
| DiffuLlamaSize=7B, Type=DD2026.01 | 0.587 | |
| A3Size=8B, Type=A32026.01 | 0.584 | |
| PCMind-2.1-Kaiyuan-2BParams=2B2025.12 | 0.5813 | |
| Zero-shotmode=0-S2026.02 | 0.508 | |
| A3Size=3B, Type=A32026.01 | 0.496 | |
| Qwen3-0.6BParams=0.6B2025.12 | 0.4851 | |
| EMR-MERGINGValidation=false2024.05 | 0.447 | |
| Individual2024.05 | 0.444 | |
| Ties-MergingValidation=true2024.05 | 0.428 | |
| Ties-MergingValidation=false2024.05 | 0.426 | |
| Traditional MTL2024.05 | 0.423 | |
| Fisher MergingValidation=true2024.05 | 0.422 | |
| Task ArithmeticValidation=true2024.05 | 0.415 | |
| A3Size=1B, Type=A32026.01 | 0.402 | |
| RegMeanValidation=true2024.05 | 0.401 | |
| Weight AveragingValidation=false2024.05 | 0.401 | |
| PlaidSize=1B, Type=CD2026.01 | 0.393 | |
| Task ArithmeticValidation=false2024.05 | 0.314 | |
| DreamSize=7B, Type=DD2026.01 | 0.269 |