Chinese General Knowledge on CMMLU
90.9AccuracyKimi-K2 Base
Evaluation results
| Method | Links | |
|---|---|---|
| Kimi-K2 Base# Shots=5-shot, # Activated Params=32B, # Total Params=1043B2026.01 | 90.9 | |
| DeepSeek-V3.2 Exp Base# Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.01 | 88.9 | |
| DeepSeek-V3.1 Base# Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.01 | 88.8 | |
| MiMo-V2-Flash Base# Shots=5-shot, # Activated Params=15B, # Total Params=309B2026.01 | 87.4 | |
| Qwen3.5-9BShots=52026.04 | 84.35 | |
| Qwen3-14BShots=52026.04 | 82.1 | |
| Qwen3-8BShots=52026.04 | 77.45 | |
| XekRung-8BShots=52026.04 | 77.05 | |
| SecGPT-14BShots=52026.04 | 72.35 | |
| Llama-3.3-70B-InstructShots=52026.04 | 72.25 | |
| DeepSeekV2-LiteK=6, Avg. K=6.002026.05 | 60.93 | |
| BEAMSparsity Level=High Sparsity, beta=0.1, Avg. K=1.082026.05 | 60.67 | |
| BEAMSparsity Level=Mid Sparsity, beta=0.01, Avg. K=2.612026.05 | 60.42 | |
| Top-K ReducedSparsity Level=Mid Sparsity, K=4, Avg. K=4.002026.05 | 60.4 | |
| Top-K ReducedSparsity Level=High Sparsity, K=2, Avg. K=2.002026.05 | 60.39 | |
| Top-K PruningSparsity Level=Mid Sparsity, K=4, Avg. K=4.002026.05 | 60 | |
| AdaMoESparsity Level=Mid Sparsity, Null=64, Avg. K=3.252026.05 | 58.85 | |
| MoE-DynamicSparsity Level=High Sparsity, phi=0.1, Avg. K=3.902026.05 | 58.43 | |
| MoE-DynamicSparsity Level=Mid Sparsity, phi=0.3, Avg. K=4.312026.05 | 58.12 | |
| BEAMSparsity Level=Extreme Sparsity, beta=1.0, Avg. K=0.482026.05 | 54.48 | |
| AdaMoESparsity Level=High Sparsity, Null=128, Avg. K=2.112026.05 | 52.29 | |
| Top-K ReducedSparsity Level=Extreme Sparsity, K=1, Avg. K=1.002026.05 | 45.01 | |
| Llama-Primus-Reasoning-8BShots=52026.04 | 43.15 | |
| Top-K PruningSparsity Level=High Sparsity, K=2, Avg. K=2.002026.05 | 36.91 | |
| Llama-3.1-8B-InstructShots=52026.04 | 34.45 |