List your company
ResearchFormerly WizWand

Chinese Language Understanding on C-Eval

92.5AccuracyKimi-K2
20.63639.29357.9576.607Jul 15, 2024Nov 5, 2024Feb 26, 2025Jun 19, 2025Oct 10, 2025Jan 31, 2026May 24, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Kimi-K2Model Variant=Base, # Shots=5-shot, # Activated Params=32B, # Total Params=1043B2026.0292.5
Qwen2-72B2024.0791
DeepSeek V3.2Model Variant=Exp Base, # Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.0291
DeepSeek V3.1Model Variant=Base, # Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.0290
Step 3.5 FlashModel Variant=Base, # Shots=5-shot, # Activated Params=11B, # Total Params=196B2026.0289.6
Qwen1.5-110B2024.0789.1
MiMo-V2 FlashModel Variant=Base, # Shots=5-shot, # Activated Params=15B, # Total Params=309B2026.0287.9
GLM-4.5Model Variant=Base, # Shots=5-shot, # Activated Params=32B, # Total Params=355B2026.0286.9
Qwen1.5-72B2024.0784.1
Qwen2.5 (base)Params=7B, Tokens=18T, Complexity Type=Quadratic, Evaluation Framework=HuggingFace, Evaluation Protocol=perplexity-based2025.0981.6
Qwen3-4BParams=4B2025.1278.5
Qwen2.5-3BParams=3B2025.1274.65
SDAR 8BParameters=8B, Sampling=Standard, Note=Non-official replication2025.1272.7
NBDiff-7B-INSTRUCTParameters=7B, Training Protocol=Instruct, Sampling=Standard2025.1272.5
Qwen2-1.5BParams=1.5B2025.1271.29
Qwen2-1.5B# Non-Emb Params=1.2B2024.0770.6
SpikingBrain-7BParams=7B, Tokens=+150B, Complexity Type=Linear, Evaluation Framework=HuggingFace, Evaluation Protocol=perplexity-based2025.0969.8
Qwen2.5-1.5BParams=1.5B2025.1268.63
Qwen3-1.7BParams=1.7B2025.1266.7
LLaDA2.0-mini preview 16BA1BParameters=16BA1B, Sampling=Standard2025.1266.5
HSA-ULTraining Strategy=Annealing, Architecture=MoE, Total Params=8B, Activated Params=1B, Training Tokens=8T2025.1165.98
CAMELTraining Objective=Balanced, Sampling Strategy=Hourglass2026.0365.3
Llama-3-70B2024.0765.2
HySparse# Shots=5-shot, Model Architecture=80B MoE (Hybrid 1:11), Attention Variant=HySparse2026.0265
Full-Attn# Shots=5-shot, Model Architecture=80B MoE (Hybrid 1:11), Attention Variant=Full-Attn2026.0264.6
LLaDA-MoE 7B-A1BParameters=7B-A1B, Sampling=Standard2025.1263.9
Human DesignedTraining Objective=Balanced, Sampling Strategy=Rectangle2026.0363.7
OPTIMERBase Model=Gemma 3 27B2026.0363.52
DMLTraining Objective=Balanced, Sampling Strategy=Rectangle2026.0363.1
SODMTraining Objective=Balanced, Sampling Strategy=Rectangle2026.0362.8
Model-size agnosticTraining Objective=Balanced, Sampling Strategy=Rectangle2026.0362.7
OPTIMERBase Model=SEA-LION v42026.0362.56
Qwen1.5-1.8B# Non-Emb Params=1.2B2024.0759.7
Hybrid SWA# Shots=5-shot, Model Architecture=80B MoE (Hybrid 1:11), Attention Variant=Hybrid SWA2026.0258.8
HSA-ULTraining Strategy=Base, Architecture=MoE, Total Params=8B, Activated Params=1B, Training Tokens=8T2025.1158.36
Qwen2-0.5B# Non-Emb Params=0.3B2024.0758.2
Dream-v0 Instruct-7BParameters=7B, Training Protocol=Instruct, Sampling=Standard2025.1258
Qwen3-0.6BParams=0.6B2025.1257.03
TRM-MoETraining Strategy=Base, Architecture=MoE, Total Params=8B, Activated Params=1B, Training Tokens=8T2025.1156.87
Mixtral-8x22B2024.0754.6
Qwen3Training Strategy=Annealing, Architecture=Dense, Total Params=0.6B, Activated Params=0.6B, Training Tokens=36T2025.1154.57
Qwen2.5Training Strategy=Annealing, Architecture=Dense, Total Params=0.5B, Activated Params=0.5B, Training Tokens=18T2025.1154.17
YuLan-Mini-2.4BParams=2.4B2025.1252.32
HySparse# Shots=5-shot, Model Architecture=7B Dense (Hybrid 1:3), Attention Variant=HySparse2026.0252.2
Llama3.1Params=8B, Tokens=15T, Complexity Type=Quadratic, Evaluation Framework=HuggingFace, Evaluation Protocol=perplexity-based2025.0951.46
SEA-LION v4 27B IT2026.0350.89
SmolLM3-3BParams=3B2025.1250.84
Full-Attn# Shots=5-shot, Model Architecture=7B Dense (Hybrid 1:3), Attention Variant=Full-Attn2026.0250.6
Hybrid SWA# Shots=5-shot, Model Architecture=7B Dense (Hybrid 1:3), Attention Variant=Hybrid SWA2026.0250.6
MistralParams=7B, Complexity Type=Linear, Evaluation Framework=HuggingFace, Evaluation Protocol=perplexity-based2025.0947.04
PCMind-2.1-Kaiyuan-2BParams=2B2025.1246.3
llama-3.2-3BParams=3B2025.1245.67
HSA-ULTraining Strategy=Annealing, Architecture=Dense, Total Params=0.5B, Activated Params=0.5B, Training Tokens=4T2025.1144.3
Falcon-MambaParams=7B, Tokens=5.8T, Complexity Type=Linear, Evaluation Framework=HuggingFace, Evaluation Protocol=perplexity-based2025.0941.93
gemma2-2BParams=2B2025.1241.35
NITPModel scale=9bA1b, Evaluation=few-shot, Context length=81922026.0540.72
NTPModel scale=9bA1b, Evaluation=few-shot, Context length=81922026.0538.97
Gemma 3 27B IT2026.0337.96
Zamba-v1Params=7B, Tokens=1T, Complexity Type=Hybrid, Evaluation Framework=HuggingFace, Evaluation Protocol=perplexity-based2025.0936.4
SmolLM2-1.7BParams=1.7B2025.1235.06
NTPModel scale=3bA0.5b, Evaluation=few-shot, Context length=81922026.0533.53
NITPModel scale=1.9bA0.3b, Evaluation=few-shot, Context length=81922026.0533.16
NITPModel scale=3bA0.5b, Evaluation=few-shot, Context length=81922026.0533.12
NTPModel scale=1.9bA0.3b, Evaluation=few-shot, Context length=81922026.0532.31
OLMo-2-0425-1BParams=1B2025.1230.53
llama-3.2-1BParams=1B2025.1229.82
Gemma-2B# Non-Emb Params=2.0B2024.0728
Phi-2# Non-Emb Params=2.5B2024.0723.4