Commonsense Reasoning on WinoGrande (WinoG)
79.95AccuracyAUSteer
Evaluation Results
| Method | Links | |
|---|---|---|
| AUSteerModel=Llama-3.3-70B-Instruct2026.02 | 79.95 | |
| VanillaModel=Llama-3.3-70B-Instruct2026.02 | 78.14 | |
| WandaBackbone=Llama-3.1-8B-Instruct, Pruning Ratio=30%2026.02 | 74.59 | |
| POPBackbone=Gemma-3-12B-It, Pruning Ratio=33.3%2026.02 | 74.59 | |
| Full ModelBackbone=Llama-3.1-8B-Instruct, Pruning Ratio=0%2026.02 | 74.4 | |
| Full ModelBackbone=Gemma-3-12B-It, Pruning Ratio=0%2026.02 | 74.35 | |
| POPBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=33.3%2026.02 | 73.88 | |
| Full ModelBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=0%2026.02 | 73.72 | |
| WandaBackbone=Gemma-3-12B-It, Pruning Ratio=30%2026.02 | 73.64 | |
| POPBackbone=Llama-3.1-8B-Instruct, Pruning Ratio=31.25%2026.02 | 73.4 | |
| WandaBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=30%2026.02 | 72.45 | |
| ShortGPTBackbone=Llama-3.1-8B-Instruct, Pruning Ratio=25%2026.02 | 69.77 | |
| AUSteerModel=Qwen3-30B-A3B2026.02 | 67.17 | |
| SliceGPTBackbone=Llama-3.1-8B-Instruct, Pruning Ratio=25%2026.02 | 66.54 | |
| VanillaModel=Qwen3-30B-A3B2026.02 | 65.98 | |
| ShortGPTBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=25%2026.02 | 61.56 | |
| FO-LoRAModel=OPT-1.3b, Optimizer=Adam2026.04 | 60.1 | |
| OriginalModel=LLaMA-3.2-1B-Instruct, Compression Ratio=100%, Precision=fp162025.07 | 59.5 | |
| FO-LoRAModel=OPT-1.3b, Optimizer=SGD2026.04 | 59 | |
| Hybrid-LoRAModel=OPT-1.3b, Optimizer=SGD2026.04 | 58.3 | |
| SliceGPTBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=25%2026.02 | 57.93 | |
| SliceGPTBackbone=Gemma-3-12B-It, Pruning Ratio=25%2026.02 | 54.14 | |
| COALAμModel=LLaMA-3.2-1B-Instruct, Compression Ratio=90%, Precision=fp162025.07 | 54 | |
| SVD-LLMModel=LLaMA-3.2-1B-Instruct, Compression Ratio=90%, Precision=fp162025.07 | 53.8 | |
| COALAμ=0Model=LLaMA-3.2-1B-Instruct, Compression Ratio=90%, Precision=fp162025.07 | 53.2 | |
| RegMixBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 53.1 | |
| DoReMiBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 52.2 | |
| CCDD-MoEDiT# params.=104.0M, Evaluation Protocol=Zero-shot2025.10 | 51.93 | |
| GPT2# params.=110M, Evaluation Protocol=Zero-shot2025.10 | 51.62 | |
| ASVDModel=LLaMA-3.2-1B-Instruct, Compression Ratio=90%, Precision=fp162025.07 | 51.3 | |
| DoGraphBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 51.2 | |
| DoGraphBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 50.8 | |
| Llama (retrain)# params.=117M, Evaluation Protocol=Zero-shot2025.10 | 50.57 | |
| UniformBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 50.5 | |
| Data Mixing LawBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 50.4 | |
| DOGEBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 50.4 | |
| Dynamic Loss-BasedBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 50.1 | |
| RegMixBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 50 | |
| CCDD-MMDiT# params.=144.1M, Evaluation Protocol=Zero-shot2025.10 | 49.96 | |
| UniformBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 49.9 | |
| DoReMiBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 49.9 | |
| DOGEBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 49.8 | |
| RegMixModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 49.8 | |
| DoGraphModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 49.7 | |
| GIDD# params.=92.1M, Evaluation Protocol=Zero-shot2025.10 | 49.49 | |
| MDLM# params.=92.1M, Evaluation Protocol=Zero-shot2025.10 | 49.41 | |
| Dynamic Loss-BasedBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 49.3 | |
| UniformModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 49.3 | |
| Dynamic Loss-BasedModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 49.2 | |
| Data Mixing LawBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 49.1 | |
| DOGEModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 49 | |
| Data Mixing LawModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 48.9 | |
| DoReMiModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 48.8 | |
| ShortGPTBackbone=Gemma-3-12B-It, Pruning Ratio=25%2026.02 | 48.7 | |
| GIDD-base# params.=320M, Evaluation Protocol=Zero-shot2025.10 | 48.3 |