Reasoning on BBH 3-shot
65.69BBH 3-shot ScoreBaseline
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| BaselineModel=Dream-v0-Instruct-7B2026.03 | 65.69 | 1.61 | |
| Baseline+ParallelModel=Dream-v0-Instruct-7B2026.03 | 60.76 | 25.42 | |
| BaselineModel=LLaDA-8B-Instruct2026.03 | 59.58 | 1.35 | |
| d2CacheModel=Dream-v0-Instruct-7B2026.03 | 59.51 | 99.6 | |
| Fast-dLLM (Dual)Model=Dream-v0-Instruct-7B2026.03 | 58.09 | 40.96 | |
| Baseline+ParallelModel=LLaDA-8B-Instruct2026.03 | 57.9 | 21.45 | |
| EntropyCacheModel=Dream-v0-Instruct-7B2026.03 | 55.86 | 155.3 | |
| d2CacheModel=LLaDA-8B-Instruct2026.03 | 55.68 | 83.94 | |
| MARS-7BModel Scale=7B, Block Size (B)=4, One-token mode (τ=1.0)=true2026.04 | 54.6 | — | |
| AR SFTModel Scale=7B, One-token mode (τ=1.0)=true2026.04 | 54 | — | |
| Elastic-CacheModel=LLaDA-8B-Instruct2026.03 | 52.57 | 42.39 | |
| Fast-dLLM (Dual)Model=LLaDA-8B-Instruct2026.03 | 52.48 | 36.89 | |
| EntropyCacheModel=LLaDA-8B-Instruct2026.03 | 52.48 | 144.9 | |
| UltraMix-190kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-190k2025.11 | 51.76 | — | |
| UltraMix-187kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-187k2025.11 | 51.67 | — | |
| UltraMix-170kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-170k2025.11 | 50.96 | — | |
| TuluDPOBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.11 | 48.08 | — | |
| SFTBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=SFT2025.11 | 47.93 | — | |
| UltraMix-190kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-190k2025.11 | 44.96 | — | |
| Instella-3B-SFT (UM-190k)DPO Training Dataset=UM-190k, Evaluation Protocol=3-shot2025.11 | 44.95 | — | |
| Instella-3B-SFT (UM-187k)DPO Training Dataset=UM-187k, Evaluation Protocol=3-shot2025.11 | 44.89 | — | |
| UltraMix-170kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-170k2025.11 | 44.68 | — | |
| Instella-3B-SFT (UM-170k)DPO Training Dataset=UM-170k, Evaluation Protocol=3-shot2025.11 | 44.62 | — | |
| UltraMix-187kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-187k2025.11 | 44.21 | — | |
| Instella-3B-SFT (TuluDPO)DPO Training Dataset=TuluDPO, Evaluation Protocol=3-shot2025.11 | 43.39 | — | |
| Instella-3B-SFT (ORPO)DPO Training Dataset=ORPO, Evaluation Protocol=3-shot2025.11 | 43.19 | — | |
| Instella-3B-SFT (UltraFB)DPO Training Dataset=UltraFB, Evaluation Protocol=3-shot2025.11 | 43.05 | — | |
| Instella-3B-SFT (HelpSteer)DPO Training Dataset=HelpSteer, Evaluation Protocol=3-shot2025.11 | 42.87 | — | |
| Instella-3B-SFT (SFT)DPO Training Dataset=None (SFT Baseline), Evaluation Protocol=3-shot2025.11 | 42.68 | — | |
| Instella-3B-SFT (CodePref)DPO Training Dataset=CodePref, Evaluation Protocol=3-shot2025.11 | 42.39 | — | |
| UM-190kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-190k2025.11 | 41.09 | — | |
| UM-187kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-187k2025.11 | 40.83 | — | |
| TuluDPOBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.11 | 40.46 | — | |
| UM-170kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-170k2025.11 | 40.45 | — | |
| ORPOBase Model=OLMo-2-7B-SFT, DPO Training Dataset=ORPO2025.11 | 39.65 | — | |
| UltraFBBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UltraFB2025.11 | 39.44 | — | |
| TuluDPOBase Model=OLMo-2-7B-SFT, DPO Training Dataset=TuluDPO2025.11 | 39.26 | — | |
| SFTBase Model=OLMo-2-7B-SFT, DPO Training Dataset=SFT2025.11 | 38.88 | — | |
| HelpSteerBase Model=OLMo-2-7B-SFT, DPO Training Dataset=HelpSteer2025.11 | 38.85 | — | |
| SFTBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=SFT2025.11 | 38.59 | — | |
| CodePrefBase Model=OLMo-2-7B-SFT, DPO Training Dataset=CodePref2025.11 | 38.15 | — | |
| Elastic-CacheModel=Dream-v0-Instruct-7B2026.03 | 31.52 | 52.61 | |
| MARS-0.5B w/o SFT lossModel Scale=0.5B, Block Size (B)=4, SFT Loss Configuration=False, One-token mode (τ=1.0)=true2026.04 | 27.4 | — | |
| MARS-0.5BModel Scale=0.5B, Block Size (B)=4, SFT Loss Configuration=True, One-token mode (τ=1.0)=true2026.04 | 26.6 | — | |
| AR SFT (5 ep)Model Scale=0.5B, Training Epochs=5, One-token mode (τ=1.0)=true2026.04 | 26.3 | — | |
| AR SFT (10 ep)Model Scale=0.5B, Training Epochs=10, One-token mode (τ=1.0)=true2026.04 | 26.3 | — | |
| Base (no fine-tune)Model Scale=7B, One-token mode (τ=1.0)=true2026.04 | 24 | — | |
| Block DiffusionModel Scale=0.5B, Block Size (B)=4, One-token mode (τ=1.0)=true2026.04 | 7.5 | — | |
| Base (no fine-tune)Model Scale=0.5B, One-token mode (τ=1.0)=true2026.04 | 6.1 | — |