Language Understanding on CEval
83.56AccuracyQwen3-30B-A3B
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen3-30B-A3BModel=Qwen3-30B-A3B, Target Top-K (K)=8, Avg. K=8.002026.05 | 83.56 | |
| BEAMModel=Qwen3-30B-A3B, Beta (β)=0.01, Avg. K=4.232026.05 | 81.46 | |
| Qwen1.5-MoEModel=Qwen1.5-MoE-A2.7B, Target Top-K (K)=4, Avg. K=4.002026.05 | 74.15 | |
| BEAMModel=Qwen1.5-MoE-A2.7B, Beta (β)=0.01, Avg. K=1.562026.05 | 70.47 | |
| FPBackbone=LLADA-1.5-8B, Weight Bits=Full Precision, Activation Bits=Full Precision2026.06 | 70.05 | |
| FPBackbone=LLADA-8B, Weight Bits=Full Precision, Activation Bits=Full Precision2026.06 | 69.54 | |
| Qwen1.5-7BArchitecture=Dense, Decoding Framework=vLLM v0.8.4, Precision=bfloat16, max_model_len=4096, temperature=0.1, seed=12342026.06 | 67.9 | |
| openPangu-Embedded RL# Total Params=1B, Training Stage=RL2026.05 | 67.13 | |
| openPangu-Embedded KD# Total Params=1B, Training Stage=KD (NPD)2026.05 | 66.73 | |
| MiniCPM4# Total Params=0.5B2026.05 | 66.11 | |
| FPBackbone=DREAM-7B, Weight Bits=Full Precision, Activation Bits=Full Precision2026.06 | 64.89 | |
| STaR-QuantBackbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.06 | 64.56 | |
| STaR-QuantBackbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.06 | 64.34 | |
| Qwen2.52025.12 | 63.03 | |
| Fisher-MoEArchitecture=Sparse MoE (compressed), Compression Ratio=50%, Calibration Method=128 GSM8K calibration samples, Decoding Framework=vLLM v0.8.4, Precision=bfloat16, max_model_len=4096, temperature=0.1, seed=12342026.06 | 62 | |
| DLLMQuant+Backbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.06 | 61.28 | |
| DLLMQuant++Backbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.06 | 61.22 | |
| Qwen3# Total Params=1.7B2026.05 | 61 | |
| DLLMQuant++Backbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.06 | 60.04 | |
| AWQBackbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.06 | 60.03 | |
| DLLMQuant+Backbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.06 | 59.38 | |
| Qwen2.5# Total Params=1.5B2026.05 | 59.3 | |
| Qwen1.5-MoE-A2.7BArchitecture=Sparse MoE, Decoding Framework=vLLM v0.8.4, Precision=bfloat16, max_model_len=4096, temperature=0.1, seed=12342026.06 | 59.3 | |
| Qwen32025.12 | 59.16 | |
| RTNBackbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.06 | 58.96 | |
| STaR-QuantBackbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.06 | 58.79 | |
| openPangu-Embedded SFT# Total Params=1B, Training Stage=SFT2026.05 | 58.51 | |
| AWQBackbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.06 | 58.43 | |
| RTNBackbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.06 | 57.95 | |
| QuaRotBackbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.06 | 57.66 | |
| QuaRotBackbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.06 | 56.32 | |
| DLLMQuant++Backbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.06 | 55.89 | |
| DeepSeekV2-LiteModel=DeepSeekV2-Lite, Target Top-K (K)=6, Avg. K=6.002026.05 | 55.26 | |
| DLLMQuant+Backbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.06 | 54.21 | |
| BEAMModel=DeepSeekV2-Lite, Beta (β)=0.01, Avg. K=2.612026.05 | 54.12 | |
| AWQBackbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.06 | 53.27 | |
| QuaRotBackbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.06 | 53.19 | |
| Qwen3-4B2026.02 | 50.81 | |
| RTNBackbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.06 | 49.89 | |
| PretrainRLBackbone=Qwen3-4B2026.02 | 49.06 | |
| Gamayun2025.12 | 44.81 | |
| Qwen3# Total Params=0.6B2026.05 | 42.6 | |
| Llama3.22025.12 | 41.74 | |
| NITPModel Scale=3B2026.05 | 40.14 | |
| NTPModel Scale=3B2026.05 | 39.17 | |
| NITPModel Scale=2B2026.05 | 37.81 | |
| DynMoEModel=Qwen1.5-MoE-A2.7B, Avg. K=30.062026.05 | 37.6 | |
| DynMoEModel=Qwen3-30B-A3B, Avg. K=61.662026.05 | 36.42 | |
| NTPModel Scale=2B2026.05 | 35.67 | |
| Gemma32025.12 | 34.83 | |
| Expert Divergence LearningModel Size=8B-A0.8B, Scheme=49-class2026.02 | 33.81 | |
| Expert Divergence LearningModel Size=3B-A0.3B, Scheme=49-class2026.02 | 33.75 | |
| NTPModel Scale=0.5B2026.05 | 33.72 | |
| Expert Divergence LearningModel Size=15B-A1.5B, Scheme=49-class2026.02 | 33.45 | |
| Expert Divergence LearningModel Size=8B-A0.8B, Scheme=3-class2026.02 | 33.27 | |
| MoEModel Size=8B-A0.8B, Training Strategy=Baseline2026.02 | 32.88 | |
| MoEModel Size=3B-A0.3B, Training Strategy=Baseline2026.02 | 32.87 | |
| MoEModel Size=15B-A1.5B, Training Strategy=Baseline2026.02 | 32.8 | |
| NITPModel Scale=0.5B2026.05 | 32.78 | |
| Expert Divergence LearningModel Size=15B-A1.5B, Scheme=3-class2026.02 | 32.53 | |
| Gemma3# Total Params=1B2026.05 | 32.49 | |
| Expert Divergence LearningModel Size=3B-A0.3B, Scheme=3-class2026.02 | 32.08 | |
| Llama3.2# Total Params=1B2026.05 | 32.08 | |
| SFTBackbone=Qwen3-4B2026.02 | 29.38 | |
| MiniMind-3Params=26M2026.05 | 24.5 | |
| MiniWinParams=26M2026.05 | 23.8 | |
| DynMoEModel=DeepSeekV2-Lite, Avg. K=30.502026.05 | 6.63 |