List your company
ResearchFormerly WizWand

Commonsense Reasoning on WinoGrande (WinoG)

79.95AccuracyAUSteer
47.03455.579564.12572.6705Jul 10, 2025Aug 24, 2025Oct 9, 2025Nov 24, 2025Jan 8, 2026Feb 23, 2026Apr 10, 2026
Updated 5mo ago

Evaluation results

MethodLinks
AUSteerModel=Llama-3.3-70B-Instruct2026.0279.95
VanillaModel=Llama-3.3-70B-Instruct2026.0278.14
WandaBackbone=Llama-3.1-8B-Instruct, Pruning Ratio=30%2026.0274.59
POPBackbone=Gemma-3-12B-It, Pruning Ratio=33.3%2026.0274.59
Full ModelBackbone=Llama-3.1-8B-Instruct, Pruning Ratio=0%2026.0274.4
Full ModelBackbone=Gemma-3-12B-It, Pruning Ratio=0%2026.0274.35
POPBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=33.3%2026.0273.88
Full ModelBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=0%2026.0273.72
WandaBackbone=Gemma-3-12B-It, Pruning Ratio=30%2026.0273.64
POPBackbone=Llama-3.1-8B-Instruct, Pruning Ratio=31.25%2026.0273.4
WandaBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=30%2026.0272.45
ShortGPTBackbone=Llama-3.1-8B-Instruct, Pruning Ratio=25%2026.0269.77
AUSteerModel=Qwen3-30B-A3B2026.0267.17
SliceGPTBackbone=Llama-3.1-8B-Instruct, Pruning Ratio=25%2026.0266.54
VanillaModel=Qwen3-30B-A3B2026.0265.98
ShortGPTBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=25%2026.0261.56
FO-LoRAModel=OPT-1.3b, Optimizer=Adam2026.0460.1
OriginalModel=LLaMA-3.2-1B-Instruct, Compression Ratio=100%, Precision=fp162025.0759.5
FO-LoRAModel=OPT-1.3b, Optimizer=SGD2026.0459
Hybrid-LoRAModel=OPT-1.3b, Optimizer=SGD2026.0458.3
SliceGPTBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=25%2026.0257.93
SliceGPTBackbone=Gemma-3-12B-It, Pruning Ratio=25%2026.0254.14
COALAμModel=LLaMA-3.2-1B-Instruct, Compression Ratio=90%, Precision=fp162025.0754
SVD-LLMModel=LLaMA-3.2-1B-Instruct, Compression Ratio=90%, Precision=fp162025.0753.8
COALAμ=0Model=LLaMA-3.2-1B-Instruct, Compression Ratio=90%, Precision=fp162025.0753.2
RegMixBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.0453.1
DoReMiBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.0452.2
CCDD-MoEDiT# params.=104.0M, Evaluation Protocol=Zero-shot2025.1051.93
GPT2# params.=110M, Evaluation Protocol=Zero-shot2025.1051.62
ASVDModel=LLaMA-3.2-1B-Instruct, Compression Ratio=90%, Precision=fp162025.0751.3
DoGraphBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.0451.2
DoGraphBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.0450.8
Llama (retrain)# params.=117M, Evaluation Protocol=Zero-shot2025.1050.57
UniformBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.0450.5
Data Mixing LawBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.0450.4
DOGEBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.0450.4
Dynamic Loss-BasedBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.0450.1
RegMixBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.0450
CCDD-MMDiT# params.=144.1M, Evaluation Protocol=Zero-shot2025.1049.96
UniformBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.0449.9
DoReMiBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.0449.9
DOGEBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.0449.8
RegMixModel=GPT-2 Small, Training Data=SlimPajama2026.0449.8
DoGraphModel=GPT-2 Small, Training Data=SlimPajama2026.0449.7
GIDD# params.=92.1M, Evaluation Protocol=Zero-shot2025.1049.49
MDLM# params.=92.1M, Evaluation Protocol=Zero-shot2025.1049.41
Dynamic Loss-BasedBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.0449.3
UniformModel=GPT-2 Small, Training Data=SlimPajama2026.0449.3
Dynamic Loss-BasedModel=GPT-2 Small, Training Data=SlimPajama2026.0449.2
Data Mixing LawBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.0449.1
DOGEModel=GPT-2 Small, Training Data=SlimPajama2026.0449
Data Mixing LawModel=GPT-2 Small, Training Data=SlimPajama2026.0448.9
DoReMiModel=GPT-2 Small, Training Data=SlimPajama2026.0448.8
ShortGPTBackbone=Gemma-3-12B-It, Pruning Ratio=25%2026.0248.7
GIDD-base# params.=320M, Evaluation Protocol=Zero-shot2025.1048.3