List your company
ResearchFormerly WizWand

Math Reasoning on MATH 500 (Accuracy, Avg, Delta)

97.6AccuracyPreMoE
-3.69622.60248.975.198May 23, 2025May 29, 2025Jun 5, 2025Jun 12, 2025Jun 18, 2025Jun 25, 2025Jul 2, 2025
Updated 5mo ago

Evaluation results

MethodLinks
PreMoEBase Model=DeepSeek-R1, Sparsity=50%2025.0597.676.521.01
FullBase Model=openPangu-Ultra, Sparsity=31.25%2025.0597.479.25—
EASY-EPBase Model=DeepSeek-R1, Sparsity=50%2025.0597.274.69-0.82
Act-LogitsBase Model=openPangu-Ultra, Sparsity=31.25%2025.0597.276.91-2.34
FullBase Model=Qwen3-30B-A3B, Sparsity=50%2025.0597.280.69—
EASY-EPBase Model=openPangu-Ultra, Sparsity=31.25%2025.0596.8276.45-2.8
PreMoEBase Model=openPangu-Ultra, Sparsity=31.25%2025.0596.878.45-0.8
FullBase Model=DeepSeek-R1, Sparsity=50%2025.0596.675.51—
FrequencyBase Model=openPangu-Ultra, Sparsity=31.25%2025.0596.469.67-9.58
PreMoEBase Model=Qwen3-30B-A3B, Sparsity=50%2025.0596.479.92-0.77
SEER (G)Base Model=openPangu-Ultra, Sparsity=31.25%2025.059670.93-8.32
SEER (L)Base Model=openPangu-Ultra, Sparsity=31.25%2025.0595.271.29-7.96
FrequencyBase Model=Qwen3-30B-A3B, Sparsity=50%2025.0590.454.26-32.76
FrequencyBase Model=DeepSeek-R1, Sparsity=50%2025.0588.849.93-25.58
Prefix-RFTBase Model=Qwen2.5-Math-7B2025.0788.4——
Act-LogitsBase Model=DeepSeek-R1, Sparsity=50%2025.0588.262.87-12.64
LUFFYBase Model=Qwen2.5-Math-7B2025.0787.6——
ReLIFTBase Model=Qwen2.5-Math-7B2025.0787.4——
SFT+RFTBase Model=Qwen2.5-Math-7B2025.0787.2——
RandomBase Model=openPangu-Ultra, Sparsity=31.25%2025.058550.13-29.12
RFTBase Model=Qwen2.5-Math-7B2025.0784.4——
UFTBase Model=Qwen2.5-Math-7B2025.0783.8——
SFTBase Model=Qwen2.5-Math-7B2025.0782.6——
RL w/ SFT LossBase Model=Qwen2.5-Math-7B2025.0780.4——
Oat-ZeroBase Model=Qwen2.5-Math-7B2025.0778——
SimpleRL-ZeroBase Model=Qwen2.5-Math-7B2025.0776——
EASY-EPBase Model=Qwen3-30B-A3B, Sparsity=50%2025.0558.6254.26-26.43
SEER (L)Base Model=DeepSeek-R1, Sparsity=50%2025.0554.823.56-51.95
RandomBase Model=DeepSeek-R1, Sparsity=50%2025.055426.69-48.82
SEER (G)Base Model=DeepSeek-R1, Sparsity=50%2025.055324.31-51.2
RandomBase Model=Qwen3-30B-A3B, Sparsity=50%2025.0544.813.07-67.62
Qwen2.5-Math-7B2025.0743.6——
All-LogitsBase Model=openPangu-Ultra, Sparsity=31.25%2025.05125.39-73.86
All-LogitsBase Model=DeepSeek-R1, Sparsity=50%2025.053.65.46-70.05
All-LogitsBase Model=Qwen3-30B-A3B, Sparsity=50%2025.051.60.35-80.34
SEER (L)Base Model=Qwen3-30B-A3B, Sparsity=50%2025.051.60.64-80.05
Act-LogitsBase Model=Qwen3-30B-A3B, Sparsity=50%2025.051.40.82-79.87
SEER (G)Base Model=Qwen3-30B-A3B, Sparsity=50%2025.050.20.34-80.35