List your company
ResearchFormerly WizWand

Math & Science on MATH 4-shot

69.6ScoreQwen3-8B
9.779225.309640.8456.3704Nov 14, 2025Nov 21, 2025Nov 28, 2025Dec 6, 2025Dec 13, 2025Dec 20, 2025Dec 28, 2025
Updated 8mo ago

Evaluation results

MethodLinks
Qwen3-8BInstruct Model=true, Model Category=AR Baseline, Shot Configuration=4-shot, Inference Framework=vLLM, Sampling Temperature=0.1, Max Generation Length=5122025.1269.6
WeDLM-8BInstruct Model=true, Model Category=WeDLM, Shot Configuration=4-shot, Inference Framework=vLLM, Sampling Temperature=0.1, Max Generation Length=512, Window Size (W)=6, Penalty Coefficient (λ)=0.12025.1264.8
WeDLM-7BInstruct Model=true, Model Category=WeDLM, Shot Configuration=4-shot, Inference Framework=vLLM, Sampling Temperature=0.1, Max Generation Length=512, Window Size (W)=6, Penalty Coefficient (λ)=0.12025.1255.4
UltraMix-190kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-190k2025.1149.55
UltraMix-187kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-187k2025.1148.79
UltraMix-170kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-170k2025.1147.55
Qwen2.5-7BInstruct Model=true, Model Category=AR Baseline, Shot Configuration=4-shot, Inference Framework=vLLM, Sampling Temperature=0.1, Max Generation Length=5122025.1245
SDAR-8BInstruct Model=true, Model Category=DLLM Baseline, Shot Configuration=4-shot, Inference Framework=JetEngine, Sampling Temperature=0.1, Max Generation Length=512, Window Size (W)=6, Penalty Coefficient (λ)=0.12025.1243.4
TuluDPOBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.1143.13
Dream-7BInstruct Model=true, Model Category=DLLM Baseline, Shot Configuration=4-shot, Inference Framework=dInfer, Sampling Temperature=0.1, Max Generation Length=512, Window Size (W)=6, Penalty Coefficient (λ)=0.12025.1241
SFTBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=SFT2025.1136.21
LLaDA-8BInstruct Model=true, Model Category=DLLM Baseline, Shot Configuration=4-shot, Inference Framework=dInfer, Sampling Temperature=0.1, Max Generation Length=512, Window Size (W)=6, Penalty Coefficient (λ)=0.12025.1234.2
UltraMixTraining Method=DPO, Dataset=UM-190k, Evaluation Protocol=4-shot2025.1132.5
UltraMixTraining Method=DPO, Dataset=UM-187k, Evaluation Protocol=4-shot2025.1132.07
TuluDPOTraining Method=DPO, Dataset=TuluDPO, Evaluation Protocol=4-shot2025.1131.21
UltraMixTraining Method=DPO, Dataset=UM-170k, Evaluation Protocol=4-shot2025.1130.98
ORPOTraining Method=ORPO, Dataset=ORPO, Evaluation Protocol=4-shot2025.1129.68
HelpSteerTraining Method=DPO, Dataset=HelpSteer, Evaluation Protocol=4-shot2025.1127.95
UltraFBTraining Method=DPO, Dataset=UltraFB, Evaluation Protocol=4-shot2025.1127.19
CodePrefTraining Method=DPO, Dataset=CodePref, Evaluation Protocol=4-shot2025.1124.46
SmolLM-3-3B-SFTTraining Method=SFT, Evaluation Protocol=4-shot2025.1124.04
UltraMix-190kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-190k2025.1123.56
TuluDPOBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.1122.66
UltraMix-187kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-187k2025.1122.03
UltraMix-170kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-170k2025.1121.22
Instella-3B-SFT (UM-190k)DPO Training Dataset=UM-190k, Evaluation Protocol=4-shot2025.1116.88
Instella-3B-SFT (UM-187k)DPO Training Dataset=UM-187k, Evaluation Protocol=4-shot2025.1116.47
Instella-3B-SFT (TuluDPO)DPO Training Dataset=TuluDPO, Evaluation Protocol=4-shot2025.1115.71
Instella-3B-SFT (UM-170k)DPO Training Dataset=UM-170k, Evaluation Protocol=4-shot2025.1115.56
Instella-3B-SFT (ORPO)DPO Training Dataset=ORPO, Evaluation Protocol=4-shot2025.1114.39
Instella-3B-SFT (UltraFB)DPO Training Dataset=UltraFB, Evaluation Protocol=4-shot2025.1113.52
Instella-3B-SFT (HelpSteer)DPO Training Dataset=HelpSteer, Evaluation Protocol=4-shot2025.1112.76
Instella-3B-SFT (SFT)DPO Training Dataset=None (SFT Baseline), Evaluation Protocol=4-shot2025.1112.31
Instella-3B-SFT (CodePref)DPO Training Dataset=CodePref, Evaluation Protocol=4-shot2025.1112.08
SFTBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=SFT2025.1112.08