List your company
ResearchFormerly WizWand

Mathematical Reasoning on GSM8K (Acc, TPF, AUP)

96AccuracyOrthrus-Qwen3-8B
38.38453.34268.383.258Mar 4, 2026Mar 20, 2026Apr 6, 2026Apr 22, 2026May 9, 2026May 25, 2026Jun 11, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Orthrus-Qwen3-8BParams=8.7B2026.0596——
Initial large teacherModel=Qwen2.5-7B-Instruct2026.0594.9——
COSEBackbone=Qwen3-4B2026.0594.4——
AZRBackbone=Qwen3-4B2026.0593.2——
COSEBackbone=Qwen3-0.6B2026.0593——
MAEBackbone=Qwen3-4B2026.0592.8——
RL (BF16)Model=Qwen3.5-9B, Bitwidth=BF162026.0592.57——
CGSModel pair=Qwen2.5-1.5B → Qwen2.5-32B, k=32, Chunk length (L)=202026.0692.5——
R-ZeroBackbone=Qwen3-4B2026.0592.4——
PRM guided searchModel pair=Qwen2.5-1.5B → Qwen2.5-32B, k=32, Guidance=Reasoning steps2026.0692.3——
AISModel=Qwen3.5-9B, Bitwidth=FP82026.0591.74——
SDAR-Qwen3-8BParams=8B2026.0591.7——
SDAR-Qwen3-30B-A3BParams=30B2026.0591.4——
FlashRL (TIS)Model=Qwen3.5-9B, Bitwidth=FP82026.0590.8——
LGSModel pair=Qwen2.5-1.5B → Qwen2.5-32B, k=32, Chunk length (L)=202026.0690.4——
LightningRL-8B-b32Model Category=Block-wise dLLMs, Evaluation Protocol=Zero-shot2026.0390.35.58492.4
RL (FP8 Rollout)Model=Qwen3.5-9B, Bitwidth=FP82026.0590.22——
AISModel=Qwen3-8B, Bitwidth=FP82026.0589.99——
RL (BF16)Model=Qwen3-8B, Bitwidth=BF162026.0589.84——
CGSModel pair=Qwen2.5-1.5B → Qwen2.5-32B, k=16, Chunk length (L)=202026.0689.3——
PRM guided searchModel pair=Qwen2.5-1.5B → Qwen2.5-32B, k=16, Guidance=Reasoning steps2026.0689.2——
BoostLoRA# Additional Params=122026.0489.1——
COSEBackbone=Llama-3.2-3B-Instruct2026.0589——
SDAR-8B-b32Model Category=Block-wise dLLMs, Evaluation Protocol=Zero-shot2026.0388.92.85252.5
FlashRL (TIS)Model=Qwen3-8B, Bitwidth=FP82026.0588.63——
Initial small teacherModel=Qwen2.5-3B-Instruct2026.0588.5——
RL (FP8 Rollout)Model=Qwen3-8B, Bitwidth=FP82026.0587.65——
LGSModel pair=Qwen2.5-1.5B → Qwen2.5-32B, k=16, Chunk length (L)=202026.0687.6——
COSEBackbone=Qwen2.5-3B-Instruct2026.0587.33——
TinyLoRA# Additional Params=8,0642026.0487.2——
Full FT# Additional Params=3.09B2026.0487——
MAEBackbone=Qwen2.5-3B-Instruct2026.0586.8——
TinyLoRA# Additional Params=129,0242026.0486.7——
AZRBackbone=Qwen2.5-3B-Instruct2026.0586.4——
R-ZeroBackbone=Qwen2.5-3B-Instruct2026.0586——
PRM guided searchModel pair=Qwen2.5-1.5B → Qwen2.5-32B, k=8, Guidance=Reasoning steps2026.0686——
CGSModel pair=Qwen2.5-1.5B → Qwen2.5-32B, k=8, Chunk length (L)=202026.0685.7——
MAEBackbone=Qwen3-0.6B2026.0585.6——
TinyLoRA# Additional Params=2522026.0485.4——
LGSModel pair=Qwen2.5-1.5B → Qwen2.5-32B, k=8, Chunk length (L)=202026.0685.4——
BaseBackbone=Qwen2.5-3B-Instruct2026.0585.2——
BaseBackbone=Qwen3-4B2026.0585——
R-ZeroBackbone=Qwen3-0.6B2026.0584.8——
AZRBackbone=Qwen3-0.6B2026.0584.2——
DreamModel Category=dLLMs, Evaluation Protocol=Zero-shot2026.0383.9183.9
CGSModel pair=Llama-3.2-1B → Llama-3.1-70B, k=32, Chunk length (L)=202026.0683.9——
Fast-dLLM-v2Params=7B2026.0583.7——
LLaDA-1.5Params=7B2026.0582.4——
dParallel-DreamModel Category=dLLMs, Evaluation Protocol=Zero-shot2026.0382.13.02245.7
d3LLM-DreamModel Category=dLLMs, Evaluation Protocol=Zero-shot2026.0381.44.94391.3
LGSModel pair=Llama-3.2-1B → Llama-3.1-70B, k=32, Chunk length (L)=202026.0681.2——
BaseBackbone=Qwen3-0.6B2026.0581——
TinyLoRA# Additional Params=162026.0480.9——
Dream-7BParams=7B2026.0579.3——
CGSModel pair=Llama-3.2-1B → Llama-3.1-70B, k=16, Chunk length (L)=202026.0679.3——
Fast-dLLM-DreamModel Category=dLLMs, Evaluation Protocol=Zero-shot2026.03791.44116.5
MSA-PTTraining Strategy=from-scratch sparse pretraining2026.0677.7——
Fast-dLLM-v2Model Category=Block-wise dLLMs, Evaluation Protocol=Zero-shot2026.0377.52.21156
PRM guided searchModel pair=Llama-3.2-1B → Llama-3.1-70B, k=32, Guidance=Reasoning steps2026.0677.3——
LGSModel pair=Llama-3.2-1B → Llama-3.1-70B, k=16, Chunk length (L)=202026.0677.3——
EAGLE-3 (LLaMA-3.1)Model Category=AR Models, Evaluation Protocol=Zero-shot2026.0376.65.12319
FullTraining Strategy=Full-Attention baseline2026.0676.2——
Base modelmode=zero-shot, # Additional Params=02026.0476——
MAEBackbone=Llama-3.2-3B-Instruct2026.0575.2——
Fast-dLLM-LLaDAModel Category=dLLMs, Evaluation Protocol=Zero-shot2026.0374.72.77205.8
Qwen-2.5-7B-itModel Category=AR Models, Evaluation Protocol=Zero-shot2026.0374.1174.1
R-ZeroBackbone=Llama-3.2-3B-Instruct2026.0573.8——
MSA-CPTTraining Strategy=sparse continued pretraining2026.0673.7——
D2F-LLaDAModel Category=dLLMs, Evaluation Protocol=Zero-shot2026.0373.22.88209.7
d3LLM-LLaDAModel Category=dLLMs, Evaluation Protocol=Zero-shot2026.0373.19.11637.7
LLaDAModel Category=dLLMs, Evaluation Protocol=Zero-shot2026.0372.6172.6
dParallel-LLaDAModel Category=dLLMs, Evaluation Protocol=Zero-shot2026.0372.65.14358.1
AZRBackbone=Llama-3.2-3B-Instruct2026.0572.4——
PRM guided searchModel pair=Llama-3.2-1B → Llama-3.1-70B, k=16, Guidance=Reasoning steps2026.0671.8——
CGSModel pair=Llama-3.2-1B → Llama-3.1-70B, k=8, Chunk length (L)=202026.0671.3——
LGSModel pair=Llama-3.2-1B → Llama-3.1-70B, k=8, Chunk length (L)=202026.0669.8——
PRM guided searchModel pair=Llama-3.2-1B → Llama-3.1-70B, k=8, Guidance=Reasoning steps2026.0665.4——
BaseBackbone=Llama-3.2-3B-Instruct2026.0563——
CLPDStudent Model=Qwen-0.5B, Data Ordering=expert CoT2026.0548.5——
CLPDStudent Model=Qwen-0.5B, Data Ordering=student-loss2026.0548.2——
CLPDStudent Model=Llama-1B, Data Ordering=expert CoT2026.0546.5——
CLPDStudent Model=Llama-1B, Data Ordering=student-loss2026.0546.3——
Progressive distillation onlyStudent Model=Qwen-0.5B, Teacher=small & large teacher2026.0546——
Curriculum learning onlyStudent Model=Qwen-0.5B, Teacher=small teacher, Data Ordering=expert CoT2026.0545.7——
Curriculum learning onlyStudent Model=Qwen-0.5B, Teacher=small teacher, Data Ordering=student-loss2026.0545.2——
Curriculum learning onlyStudent Model=Qwen-0.5B, Teacher=large teacher, Data Ordering=expert CoT2026.0545.1——
Curriculum learning onlyStudent Model=Qwen-0.5B, Teacher=large teacher, Data Ordering=student-loss2026.0545.1——
Standard distillationStudent Model=Qwen-0.5B, Teacher=small teacher2026.0544.9——
Progressive distillation onlyStudent Model=Llama-1B, Teacher=small & large teacher2026.0544.9——
Standard distillationStudent Model=Qwen-0.5B, Teacher=large teacher2026.0544.6——
Curriculum learning onlyStudent Model=Llama-1B, Teacher=small teacher, Data Ordering=expert CoT2026.0544.6——
Curriculum learning onlyStudent Model=Llama-1B, Teacher=large teacher, Data Ordering=student-loss2026.0544.4——
Curriculum learning onlyStudent Model=Llama-1B, Teacher=large teacher, Data Ordering=expert CoT2026.0544——
Curriculum learning onlyStudent Model=Llama-1B, Teacher=small teacher, Data Ordering=student-loss2026.0543.8——
Standard distillationStudent Model=Llama-1B, Teacher=small teacher2026.0542.7——
Standard distillationStudent Model=Llama-1B, Teacher=large teacher2026.0542.4——
Initial studentStudent Model=Qwen-0.5B2026.0541.4——
Initial studentStudent Model=Llama-1B2026.0540.6——