Commonsense Reasoning on WinoGrande (5-shot)
92.66AccuracyERNIE 5.0-Base
Evaluation Results
| Method | Links | |
|---|---|---|
| ERNIE 5.0-BaseNumber of shots=5-shot2026.02 | 92.66 | |
| DS V3.2-Exp-BaseNumber of shots=5-shot2026.02 | 88.87 | |
| Kimi K2-BaseNumber of shots=5-shot2026.02 | 87.61 | |
| UM-190kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-190k2025.11 | 78.03 | |
| UM-187kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-187k2025.11 | 77.95 | |
| UM-170kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-170k2025.11 | 77.54 | |
| Uniform Regularized LoopingModel=Gemma, Size=9B, Shots=5-shot2026.02 | 77.43 | |
| BaselineModel=Gemma, Size=9B, Shots=5-shot2026.02 | 77.35 | |
| Moving Average Regularized LoopingModel=Gemma, Size=9B, Shots=5-shot2026.02 | 77.35 | |
| Auto-Align Regularized LoopingModel=Gemma, Size=9B, Shots=5-shot2026.02 | 77.35 | |
| Noise AblationModel=Gemma, Size=9B, Shots=5-shot2026.02 | 77.35 | |
| FP16Model=LLaMA-3-8B, Quantization=FP162026.04 | 77.3 | |
| TuluDPOBase Model=OLMo-2-7B-SFT, DPO Training Dataset=TuluDPO2025.11 | 77.19 | |
| UltraMix-190kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-190k2025.11 | 77.06 | |
| ORPOBase Model=OLMo-2-7B-SFT, DPO Training Dataset=ORPO2025.11 | 76.92 | |
| COVERCALModel=LLaMA-3-8B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 76.9 | |
| FP16Model=Mistral-7B, Quantization=FP162026.04 | 76.8 | |
| UltraFBBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UltraFB2025.11 | 76.66 | |
| Uniform Regularized LoopingModel=Llama, Size=8B, Shots=5-shot2026.02 | 76.64 | |
| SFTBase Model=OLMo-2-7B-SFT, DPO Training Dataset=SFT2025.11 | 76.56 | |
| Moving Average Regularized LoopingModel=Llama, Size=8B, Shots=5-shot2026.02 | 76.4 | |
| SFTBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=SFT2025.11 | 76.4 | |
| HelpSteerBase Model=OLMo-2-7B-SFT, DPO Training Dataset=HelpSteer2025.11 | 76.4 | |
| UltraMix-187kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-187k2025.11 | 76.38 | |
| COVERCALModel=Mistral-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 76.3 | |
| BaselineModel=Llama, Size=8B, Shots=5-shot2026.02 | 76.16 | |
| Max-ActVarModel=LLaMA-3-8B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 76.1 | |
| Auto-Align Regularized LoopingModel=Llama, Size=8B, Shots=5-shot2026.02 | 76.09 | |
| StratifiedModel=LLaMA-3-8B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 76 | |
| Noise AblationModel=Llama, Size=8B, Shots=5-shot2026.02 | 75.93 | |
| RandomModel=LLaMA-3-8B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 75.8 | |
| Max-ActVarModel=Mistral-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 75.6 | |
| Max-PPLModel=LLaMA-3-8B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 75.5 | |
| StratifiedModel=Mistral-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 75.5 | |
| RandomModel=Mistral-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 75.4 | |
| CodePrefBase Model=OLMo-2-7B-SFT, DPO Training Dataset=CodePref2025.11 | 75.35 | |
| TuluDPOBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.11 | 75.3 | |
| Max-PPLModel=Mistral-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 75.2 | |
| UltraMix-170kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-170k2025.11 | 74.98 | |
| UltraMix-190kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-190k2025.11 | 74.64 | |
| UltraMix-187kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-187k2025.11 | 74.59 | |
| TuluDPOTraining Method=DPO, Dataset=TuluDPO, Evaluation Protocol=5-shot2025.11 | 74.01 | |
| UltraMix-170kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-170k2025.11 | 73.69 | |
| UltraMixTraining Method=DPO, Dataset=UM-190k, Evaluation Protocol=5-shot2025.11 | 73.38 | |
| SFTBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=SFT2025.11 | 73.17 | |
| UltraMixTraining Method=DPO, Dataset=UM-187k, Evaluation Protocol=5-shot2025.11 | 73.14 | |
| TuluDPOBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.11 | 72.96 | |
| Instella-3B-SFT (UM-190k)DPO Training Dataset=UM-190k, Evaluation Protocol=5-shot2025.11 | 72.89 | |
| Instella-3B-SFT (UM-187k)DPO Training Dataset=UM-187k, Evaluation Protocol=5-shot2025.11 | 72.69 | |
| UltraMixTraining Method=DPO, Dataset=UM-170k, Evaluation Protocol=5-shot2025.11 | 72.35 | |
| Instella-3B-SFT (UM-170k)DPO Training Dataset=UM-170k, Evaluation Protocol=5-shot2025.11 | 72.22 | |
| ORPOTraining Method=ORPO, Dataset=ORPO, Evaluation Protocol=5-shot2025.11 | 72.22 | |
| FP16Model=LLaMA-2-7B, Quantization=FP162026.04 | 72.1 | |
| Instella-3B-SFT (TuluDPO)DPO Training Dataset=TuluDPO, Evaluation Protocol=5-shot2025.11 | 72.05 | |
| Instella-3B-SFT (ORPO)DPO Training Dataset=ORPO, Evaluation Protocol=5-shot2025.11 | 71.86 | |
| Instella-3B-SFT (SFT)DPO Training Dataset=None (SFT Baseline), Evaluation Protocol=5-shot2025.11 | 71.82 | |
| SmolLM-3-3B-SFTTraining Method=SFT, Evaluation Protocol=5-shot2025.11 | 71.74 | |
| COVERCALModel=LLaMA-2-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 71.6 | |
| Instella-3B-SFT (HelpSteer)DPO Training Dataset=HelpSteer, Evaluation Protocol=5-shot2025.11 | 71.19 | |
| Max-ActVarModel=LLaMA-2-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 71 | |
| StratifiedModel=LLaMA-2-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 70.9 | |
| RandomModel=LLaMA-2-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 70.8 | |
| Instella-3B-SFT (UltraFB)DPO Training Dataset=UltraFB, Evaluation Protocol=5-shot2025.11 | 70.64 | |
| HelpSteerTraining Method=DPO, Dataset=HelpSteer, Evaluation Protocol=5-shot2025.11 | 70.56 | |
| Max-PPLModel=LLaMA-2-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 70.5 | |
| UltraFBTraining Method=DPO, Dataset=UltraFB, Evaluation Protocol=5-shot2025.11 | 69.54 | |
| Uniform Regularized LoopingModel=Gemma, Size=2B, Shots=5-shot2026.02 | 69.53 | |
| Moving Average Regularized LoopingModel=Gemma, Size=2B, Shots=5-shot2026.02 | 69.06 | |
| Auto-Align Regularized LoopingModel=Gemma, Size=2B, Shots=5-shot2026.02 | 68.98 | |
| BaselineModel=Gemma, Size=2B, Shots=5-shot2026.02 | 68.75 | |
| Noise AblationModel=Gemma, Size=2B, Shots=5-shot2026.02 | 68.51 | |
| UM-190kTraining Dataset=UM-190k2025.11 | 67.56 | |
| UM-187kTraining Dataset=UM-187k2025.11 | 67.29 | |
| UM-170kTraining Dataset=UM-170k2025.11 | 66.75 | |
| HelpSteerTraining Dataset=HelpSteer2025.11 | 66.54 | |
| MoE 128e8a1sModel=MoE 128e8a1s, Training Steps=100k2026.05 | 66.5 | |
| Instella-3B-SFT (CodePref)DPO Training Dataset=CodePref, Evaluation Protocol=5-shot2025.11 | 66.46 | |
| TuluDPOTraining Dataset=TuluDPO2025.11 | 66.17 | |
| ORPOTraining Dataset=ORPO2025.11 | 66.14 | |
| CodePrefTraining Method=DPO, Dataset=CodePref, Evaluation Protocol=5-shot2025.11 | 66.12 | |
| SFTTraining Dataset=SFT2025.11 | 65.67 | |
| MoE 128e8a4g1sModel=MoE 128e8a4g1s, Training Steps=100k2026.05 | 65.2 | |
| CodePrefTraining Dataset=CodePref2025.11 | 64.38 | |
| UltraFBTraining Dataset=UltraFB2025.11 | 64.09 | |
| DenseModel=Dense, Training Steps=100k2026.05 | 59 |