List your company
ResearchFormerly WizWand

Science Question Answering on GPQA (accuracy, Avg, Delta)

79.29AccuracyAct-Logits
-3.171618.236739.64561.0533May 23, 2025
Updated 7mo ago

Evaluation results

MethodLinks
Act-LogitsBase Model=openPangu-Ultra, Sparsity=31.25%2025.0579.2976.91-2.34
FullBase Model=openPangu-Ultra, Sparsity=31.25%2025.0576.7779.25—
EASY-EPBase Model=openPangu-Ultra, Sparsity=31.25%2025.0576.7476.45-2.8
PreMoEBase Model=openPangu-Ultra, Sparsity=31.25%2025.0575.7678.45-0.8
FullBase Model=DeepSeek-R1, Sparsity=50%2025.0573.2375.51—
PreMoEBase Model=DeepSeek-R1, Sparsity=50%2025.0572.2276.521.01
EASY-EPBase Model=DeepSeek-R1, Sparsity=50%2025.0570.1274.69-0.82
FullBase Model=Qwen3-30B-A3B, Sparsity=50%2025.0568.6980.69—
PreMoEBase Model=Qwen3-30B-A3B, Sparsity=50%2025.0568.1879.92-0.77
SEER (L)Base Model=openPangu-Ultra, Sparsity=31.25%2025.0562.6371.29-7.96
RandomBase Model=openPangu-Ultra, Sparsity=31.25%2025.0560.150.13-29.12
SEER (G)Base Model=openPangu-Ultra, Sparsity=31.25%2025.0557.5870.93-8.32
FrequencyBase Model=openPangu-Ultra, Sparsity=31.25%2025.0552.0269.67-9.58
EASY-EPBase Model=Qwen3-30B-A3B, Sparsity=50%2025.0550.4854.26-26.43
Act-LogitsBase Model=DeepSeek-R1, Sparsity=50%2025.0548.4862.87-12.64
FrequencyBase Model=Qwen3-30B-A3B, Sparsity=50%2025.0547.9854.26-32.76
RandomBase Model=DeepSeek-R1, Sparsity=50%2025.0539.926.69-48.82
SEER (L)Base Model=DeepSeek-R1, Sparsity=50%2025.0536.8723.56-51.95
SEER (G)Base Model=DeepSeek-R1, Sparsity=50%2025.0536.8724.31-51.2
FrequencyBase Model=DeepSeek-R1, Sparsity=50%2025.0533.3349.93-25.58
All-LogitsBase Model=DeepSeek-R1, Sparsity=50%2025.0528.795.46-70.05
RandomBase Model=Qwen3-30B-A3B, Sparsity=50%2025.0517.6813.07-67.62
All-LogitsBase Model=openPangu-Ultra, Sparsity=31.25%2025.0516.675.39-73.86
Act-LogitsBase Model=Qwen3-30B-A3B, Sparsity=50%2025.053.540.82-79.87
All-LogitsBase Model=Qwen3-30B-A3B, Sparsity=50%2025.050.510.35-80.34
SEER (L)Base Model=Qwen3-30B-A3B, Sparsity=50%2025.0500.64-80.05
SEER (G)Base Model=Qwen3-30B-A3B, Sparsity=50%2025.0500.34-80.35