List your company
ResearchFormerly WizWand

Reasoning on BBH 3-shot

65.69BBH 3-shot ScoreBaseline
3.716419.805735.89551.9843Nov 14, 2025Dec 8, 2025Jan 1, 2026Jan 25, 2026Feb 18, 2026Mar 14, 2026Apr 8, 2026
Updated 6mo ago

Evaluation results

MethodLinks
BaselineModel=Dream-v0-Instruct-7B2026.0365.691.61
Baseline+ParallelModel=Dream-v0-Instruct-7B2026.0360.7625.42
BaselineModel=LLaDA-8B-Instruct2026.0359.581.35
d2CacheModel=Dream-v0-Instruct-7B2026.0359.5199.6
Fast-dLLM (Dual)Model=Dream-v0-Instruct-7B2026.0358.0940.96
Baseline+ParallelModel=LLaDA-8B-Instruct2026.0357.921.45
EntropyCacheModel=Dream-v0-Instruct-7B2026.0355.86155.3
d2CacheModel=LLaDA-8B-Instruct2026.0355.6883.94
MARS-7BModel Scale=7B, Block Size (B)=4, One-token mode (τ=1.0)=true2026.0454.6—
AR SFTModel Scale=7B, One-token mode (τ=1.0)=true2026.0454—
Elastic-CacheModel=LLaDA-8B-Instruct2026.0352.5742.39
Fast-dLLM (Dual)Model=LLaDA-8B-Instruct2026.0352.4836.89
EntropyCacheModel=LLaDA-8B-Instruct2026.0352.48144.9
UltraMix-190kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-190k2025.1151.76—
UltraMix-187kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-187k2025.1151.67—
UltraMix-170kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-170k2025.1150.96—
TuluDPOBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.1148.08—
SFTBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=SFT2025.1147.93—
UltraMix-190kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-190k2025.1144.96—
Instella-3B-SFT (UM-190k)DPO Training Dataset=UM-190k, Evaluation Protocol=3-shot2025.1144.95—
Instella-3B-SFT (UM-187k)DPO Training Dataset=UM-187k, Evaluation Protocol=3-shot2025.1144.89—
UltraMix-170kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-170k2025.1144.68—
Instella-3B-SFT (UM-170k)DPO Training Dataset=UM-170k, Evaluation Protocol=3-shot2025.1144.62—
UltraMix-187kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-187k2025.1144.21—
Instella-3B-SFT (TuluDPO)DPO Training Dataset=TuluDPO, Evaluation Protocol=3-shot2025.1143.39—
Instella-3B-SFT (ORPO)DPO Training Dataset=ORPO, Evaluation Protocol=3-shot2025.1143.19—
Instella-3B-SFT (UltraFB)DPO Training Dataset=UltraFB, Evaluation Protocol=3-shot2025.1143.05—
Instella-3B-SFT (HelpSteer)DPO Training Dataset=HelpSteer, Evaluation Protocol=3-shot2025.1142.87—
Instella-3B-SFT (SFT)DPO Training Dataset=None (SFT Baseline), Evaluation Protocol=3-shot2025.1142.68—
Instella-3B-SFT (CodePref)DPO Training Dataset=CodePref, Evaluation Protocol=3-shot2025.1142.39—
UM-190kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-190k2025.1141.09—
UM-187kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-187k2025.1140.83—
TuluDPOBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.1140.46—
UM-170kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-170k2025.1140.45—
ORPOBase Model=OLMo-2-7B-SFT, DPO Training Dataset=ORPO2025.1139.65—
UltraFBBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UltraFB2025.1139.44—
TuluDPOBase Model=OLMo-2-7B-SFT, DPO Training Dataset=TuluDPO2025.1139.26—
SFTBase Model=OLMo-2-7B-SFT, DPO Training Dataset=SFT2025.1138.88—
HelpSteerBase Model=OLMo-2-7B-SFT, DPO Training Dataset=HelpSteer2025.1138.85—
SFTBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=SFT2025.1138.59—
CodePrefBase Model=OLMo-2-7B-SFT, DPO Training Dataset=CodePref2025.1138.15—
Elastic-CacheModel=Dream-v0-Instruct-7B2026.0331.5252.61
MARS-0.5B w/o SFT lossModel Scale=0.5B, Block Size (B)=4, SFT Loss Configuration=False, One-token mode (τ=1.0)=true2026.0427.4—
MARS-0.5BModel Scale=0.5B, Block Size (B)=4, SFT Loss Configuration=True, One-token mode (τ=1.0)=true2026.0426.6—
AR SFT (5 ep)Model Scale=0.5B, Training Epochs=5, One-token mode (τ=1.0)=true2026.0426.3—
AR SFT (10 ep)Model Scale=0.5B, Training Epochs=10, One-token mode (τ=1.0)=true2026.0426.3—
Base (no fine-tune)Model Scale=7B, One-token mode (τ=1.0)=true2026.0424—
Block DiffusionModel Scale=0.5B, Block Size (B)=4, One-token mode (τ=1.0)=true2026.047.5—
Base (no fine-tune)Model Scale=0.5B, One-token mode (τ=1.0)=true2026.046.1—