List your company
ResearchFormerly WizWand

Commonsense Reasoning on WinoGrande (5-shot)

92.66AccuracyERNIE 5.0-Base
57.653666.741875.8384.9182Nov 14, 2025Dec 15, 2025Jan 16, 2026Feb 16, 2026Mar 20, 2026Apr 20, 2026May 22, 2026
Updated 5mo ago

Evaluation results

MethodLinks
ERNIE 5.0-BaseNumber of shots=5-shot2026.0292.66
DS V3.2-Exp-BaseNumber of shots=5-shot2026.0288.87
Kimi K2-BaseNumber of shots=5-shot2026.0287.61
UM-190kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-190k2025.1178.03
UM-187kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-187k2025.1177.95
UM-170kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-170k2025.1177.54
Uniform Regularized LoopingModel=Gemma, Size=9B, Shots=5-shot2026.0277.43
BaselineModel=Gemma, Size=9B, Shots=5-shot2026.0277.35
Moving Average Regularized LoopingModel=Gemma, Size=9B, Shots=5-shot2026.0277.35
Auto-Align Regularized LoopingModel=Gemma, Size=9B, Shots=5-shot2026.0277.35
Noise AblationModel=Gemma, Size=9B, Shots=5-shot2026.0277.35
FP16Model=LLaMA-3-8B, Quantization=FP162026.0477.3
TuluDPOBase Model=OLMo-2-7B-SFT, DPO Training Dataset=TuluDPO2025.1177.19
UltraMix-190kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-190k2025.1177.06
ORPOBase Model=OLMo-2-7B-SFT, DPO Training Dataset=ORPO2025.1176.92
COVERCALModel=LLaMA-3-8B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0476.9
FP16Model=Mistral-7B, Quantization=FP162026.0476.8
UltraFBBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UltraFB2025.1176.66
Uniform Regularized LoopingModel=Llama, Size=8B, Shots=5-shot2026.0276.64
SFTBase Model=OLMo-2-7B-SFT, DPO Training Dataset=SFT2025.1176.56
Moving Average Regularized LoopingModel=Llama, Size=8B, Shots=5-shot2026.0276.4
SFTBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=SFT2025.1176.4
HelpSteerBase Model=OLMo-2-7B-SFT, DPO Training Dataset=HelpSteer2025.1176.4
UltraMix-187kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-187k2025.1176.38
COVERCALModel=Mistral-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0476.3
BaselineModel=Llama, Size=8B, Shots=5-shot2026.0276.16
Max-ActVarModel=LLaMA-3-8B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0476.1
Auto-Align Regularized LoopingModel=Llama, Size=8B, Shots=5-shot2026.0276.09
StratifiedModel=LLaMA-3-8B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0476
Noise AblationModel=Llama, Size=8B, Shots=5-shot2026.0275.93
RandomModel=LLaMA-3-8B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0475.8
Max-ActVarModel=Mistral-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0475.6
Max-PPLModel=LLaMA-3-8B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0475.5
StratifiedModel=Mistral-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0475.5
RandomModel=Mistral-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0475.4
CodePrefBase Model=OLMo-2-7B-SFT, DPO Training Dataset=CodePref2025.1175.35
TuluDPOBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.1175.3
Max-PPLModel=Mistral-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0475.2
UltraMix-170kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-170k2025.1174.98
UltraMix-190kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-190k2025.1174.64
UltraMix-187kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-187k2025.1174.59
TuluDPOTraining Method=DPO, Dataset=TuluDPO, Evaluation Protocol=5-shot2025.1174.01
UltraMix-170kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-170k2025.1173.69
UltraMixTraining Method=DPO, Dataset=UM-190k, Evaluation Protocol=5-shot2025.1173.38
SFTBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=SFT2025.1173.17
UltraMixTraining Method=DPO, Dataset=UM-187k, Evaluation Protocol=5-shot2025.1173.14
TuluDPOBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.1172.96
Instella-3B-SFT (UM-190k)DPO Training Dataset=UM-190k, Evaluation Protocol=5-shot2025.1172.89
Instella-3B-SFT (UM-187k)DPO Training Dataset=UM-187k, Evaluation Protocol=5-shot2025.1172.69
UltraMixTraining Method=DPO, Dataset=UM-170k, Evaluation Protocol=5-shot2025.1172.35
Instella-3B-SFT (UM-170k)DPO Training Dataset=UM-170k, Evaluation Protocol=5-shot2025.1172.22
ORPOTraining Method=ORPO, Dataset=ORPO, Evaluation Protocol=5-shot2025.1172.22
FP16Model=LLaMA-2-7B, Quantization=FP162026.0472.1
Instella-3B-SFT (TuluDPO)DPO Training Dataset=TuluDPO, Evaluation Protocol=5-shot2025.1172.05
Instella-3B-SFT (ORPO)DPO Training Dataset=ORPO, Evaluation Protocol=5-shot2025.1171.86
Instella-3B-SFT (SFT)DPO Training Dataset=None (SFT Baseline), Evaluation Protocol=5-shot2025.1171.82
SmolLM-3-3B-SFTTraining Method=SFT, Evaluation Protocol=5-shot2025.1171.74
COVERCALModel=LLaMA-2-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0471.6
Instella-3B-SFT (HelpSteer)DPO Training Dataset=HelpSteer, Evaluation Protocol=5-shot2025.1171.19
Max-ActVarModel=LLaMA-2-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0471
StratifiedModel=LLaMA-2-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0470.9
RandomModel=LLaMA-2-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0470.8
Instella-3B-SFT (UltraFB)DPO Training Dataset=UltraFB, Evaluation Protocol=5-shot2025.1170.64
HelpSteerTraining Method=DPO, Dataset=HelpSteer, Evaluation Protocol=5-shot2025.1170.56
Max-PPLModel=LLaMA-2-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0470.5
UltraFBTraining Method=DPO, Dataset=UltraFB, Evaluation Protocol=5-shot2025.1169.54
Uniform Regularized LoopingModel=Gemma, Size=2B, Shots=5-shot2026.0269.53
Moving Average Regularized LoopingModel=Gemma, Size=2B, Shots=5-shot2026.0269.06
Auto-Align Regularized LoopingModel=Gemma, Size=2B, Shots=5-shot2026.0268.98
BaselineModel=Gemma, Size=2B, Shots=5-shot2026.0268.75
Noise AblationModel=Gemma, Size=2B, Shots=5-shot2026.0268.51
UM-190kTraining Dataset=UM-190k2025.1167.56
UM-187kTraining Dataset=UM-187k2025.1167.29
UM-170kTraining Dataset=UM-170k2025.1166.75
HelpSteerTraining Dataset=HelpSteer2025.1166.54
MoE 128e8a1sModel=MoE 128e8a1s, Training Steps=100k2026.0566.5
Instella-3B-SFT (CodePref)DPO Training Dataset=CodePref, Evaluation Protocol=5-shot2025.1166.46
TuluDPOTraining Dataset=TuluDPO2025.1166.17
ORPOTraining Dataset=ORPO2025.1166.14
CodePrefTraining Method=DPO, Dataset=CodePref, Evaluation Protocol=5-shot2025.1166.12
SFTTraining Dataset=SFT2025.1165.67
MoE 128e8a4g1sModel=MoE 128e8a4g1s, Training Steps=100k2026.0565.2
CodePrefTraining Dataset=CodePref2025.1164.38
UltraFBTraining Dataset=UltraFB2025.1164.09
DenseModel=Dense, Training Steps=100k2026.0559