List your company
ResearchFormerly WizWand

Language Understanding on CEval

83.56AccuracyQwen3-30B-A3B
3.552824.323945.09565.8661Dec 25, 2025Jan 20, 2026Feb 16, 2026Mar 15, 2026Apr 11, 2026May 8, 2026Jun 4, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Qwen3-30B-A3BModel=Qwen3-30B-A3B, Target Top-K (K)=8, Avg. K=8.002026.0583.56
BEAMModel=Qwen3-30B-A3B, Beta (β)=0.01, Avg. K=4.232026.0581.46
Qwen1.5-MoEModel=Qwen1.5-MoE-A2.7B, Target Top-K (K)=4, Avg. K=4.002026.0574.15
BEAMModel=Qwen1.5-MoE-A2.7B, Beta (β)=0.01, Avg. K=1.562026.0570.47
FPBackbone=LLADA-1.5-8B, Weight Bits=Full Precision, Activation Bits=Full Precision2026.0670.05
FPBackbone=LLADA-8B, Weight Bits=Full Precision, Activation Bits=Full Precision2026.0669.54
Qwen1.5-7BArchitecture=Dense, Decoding Framework=vLLM v0.8.4, Precision=bfloat16, max_model_len=4096, temperature=0.1, seed=12342026.0667.9
openPangu-Embedded RL# Total Params=1B, Training Stage=RL2026.0567.13
openPangu-Embedded KD# Total Params=1B, Training Stage=KD (NPD)2026.0566.73
MiniCPM4# Total Params=0.5B2026.0566.11
FPBackbone=DREAM-7B, Weight Bits=Full Precision, Activation Bits=Full Precision2026.0664.89
STaR-QuantBackbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.0664.56
STaR-QuantBackbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.0664.34
Qwen2.52025.1263.03
Fisher-MoEArchitecture=Sparse MoE (compressed), Compression Ratio=50%, Calibration Method=128 GSM8K calibration samples, Decoding Framework=vLLM v0.8.4, Precision=bfloat16, max_model_len=4096, temperature=0.1, seed=12342026.0662
DLLMQuant+Backbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.0661.28
DLLMQuant++Backbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.0661.22
Qwen3# Total Params=1.7B2026.0561
DLLMQuant++Backbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.0660.04
AWQBackbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.0660.03
DLLMQuant+Backbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.0659.38
Qwen2.5# Total Params=1.5B2026.0559.3
Qwen1.5-MoE-A2.7BArchitecture=Sparse MoE, Decoding Framework=vLLM v0.8.4, Precision=bfloat16, max_model_len=4096, temperature=0.1, seed=12342026.0659.3
Qwen32025.1259.16
RTNBackbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.0658.96
STaR-QuantBackbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.0658.79
openPangu-Embedded SFT# Total Params=1B, Training Stage=SFT2026.0558.51
AWQBackbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.0658.43
RTNBackbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.0657.95
QuaRotBackbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.0657.66
QuaRotBackbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.0656.32
DLLMQuant++Backbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.0655.89
DeepSeekV2-LiteModel=DeepSeekV2-Lite, Target Top-K (K)=6, Avg. K=6.002026.0555.26
DLLMQuant+Backbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.0654.21
BEAMModel=DeepSeekV2-Lite, Beta (β)=0.01, Avg. K=2.612026.0554.12
AWQBackbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.0653.27
QuaRotBackbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.0653.19
Qwen3-4B2026.0250.81
RTNBackbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.0649.89
PretrainRLBackbone=Qwen3-4B2026.0249.06
Gamayun2025.1244.81
Qwen3# Total Params=0.6B2026.0542.6
Llama3.22025.1241.74
NITPModel Scale=3B2026.0540.14
NTPModel Scale=3B2026.0539.17
NITPModel Scale=2B2026.0537.81
DynMoEModel=Qwen1.5-MoE-A2.7B, Avg. K=30.062026.0537.6
DynMoEModel=Qwen3-30B-A3B, Avg. K=61.662026.0536.42
NTPModel Scale=2B2026.0535.67
Gemma32025.1234.83
Expert Divergence LearningModel Size=8B-A0.8B, Scheme=49-class2026.0233.81
Expert Divergence LearningModel Size=3B-A0.3B, Scheme=49-class2026.0233.75
NTPModel Scale=0.5B2026.0533.72
Expert Divergence LearningModel Size=15B-A1.5B, Scheme=49-class2026.0233.45
Expert Divergence LearningModel Size=8B-A0.8B, Scheme=3-class2026.0233.27
MoEModel Size=8B-A0.8B, Training Strategy=Baseline2026.0232.88
MoEModel Size=3B-A0.3B, Training Strategy=Baseline2026.0232.87
MoEModel Size=15B-A1.5B, Training Strategy=Baseline2026.0232.8
NITPModel Scale=0.5B2026.0532.78
Expert Divergence LearningModel Size=15B-A1.5B, Scheme=3-class2026.0232.53
Gemma3# Total Params=1B2026.0532.49
Expert Divergence LearningModel Size=3B-A0.3B, Scheme=3-class2026.0232.08
Llama3.2# Total Params=1B2026.0532.08
SFTBackbone=Qwen3-4B2026.0229.38
MiniMind-3Params=26M2026.0524.5
MiniWinParams=26M2026.0523.8
DynMoEModel=DeepSeekV2-Lite, Avg. K=30.502026.056.63