List your company
ResearchFormerly WizWand

Multitask Language Understanding on MMLU-Pro

89.31AccuracyGemini-3-Pro-Preview
62.883669.744376.60583.4657Jul 31, 2024Nov 19, 2024Mar 10, 2025Jun 29, 2025Oct 18, 2025Feb 6, 2026May 28, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Gemini-3-Pro-Preview2026.0589.31
GPT-5 (High)2026.0287.1
Gemini 3-Pro2026.0286.88
GPT-5-2025-08-07Setting=high2026.0586.21
Gemini 2.5 Pro2026.0286.2
Gemini 2.5-Pro2026.0286.2
Kimi-K2.52026.0586.2
Claude Opus 4Reasoning=Thinking2026.0286
Gemini-2.5-Pro2026.0585.85
DeepSeek-V3.22026.0585.81
JTMode=Think2026.0585.57
DeepSeek-V3.2-Speciale2026.0585.45
GLM-5Precision=FP82026.0585.23
DS V3.2-Thinking2026.0285
Kimi-K2Mode=Thinking2026.0584.34
TeleChatMode=Thinking2026.0584.09
GLM-4-72026.0583.97
ERNIE 5.02026.0283.8
Claude Sonnet 42026.0283.7
Claude Sonnet 4Reasoning=Thinking2026.0283.7
Interfaze-Beta2026.0283.6
DeepSeek-R1-05282026.0583.52
Step-3.5-Flash2026.0583.46
Qwen3Parameters=235B, Active Parameters=A22B, Mode=Thinking, Version=25072026.0583.46
DeepSeek-Chat-V3-03242026.0583.27
Mimo-V2-Flash2026.0583.11
doubao-seed-1.6Mode=thinking, Version=2506152026.0583.11
o3-2025-04-162026.0583.02
Claude Sonnet 4Mode=Thinking2026.0583.02
GLM-4.62026.0582.95
GLM-4.52026.0582.72
Finix-P1-32BMode=Thinking2026.0582.65
o4-mini-2025-04-16Setting=high2026.0582.42
Qwen3-NextParameters=80B, Active Parameters=A3B, Mode=Thinking2026.0582.02
MiniMax-M2.52026.0581.74
MiniMax-M22026.0581.6
Qwen3-NextParameters=80B, Active Parameters=A3B, Type=Instruct2026.0581.31
GPT-4.1-202504142026.0581.02
LongCat-Flash-Chat2026.0580.95
Gemini 2.5 Flash2026.0280.9
Qwen3Parameters=235B, Active Parameters=A22B, Mode=Thinking2026.0580.89
GLM-4.5-Air2026.0580.68
Seed-OSS-36BType=Instruct2026.0580.62
GPT-4.12026.0280.6
GPT-5Reasoning=Minimal Reasoning2026.0280.6
PEARBackbone=Qwen-3-30B-A3B2026.0580.5
gpt-oss-120bSetting=high2026.0579.68
Llama4-MaverickParameters=17B, Experts=128E, Type=Instruct2026.0579.56
Qwen3Parameters=30B, Active Parameters=A3B, Mode=Thinking, Version=25072026.0579.46
o3-mini-2025-04-16Setting=high2026.0579.24
Qwen3Parameters=235B, Active Parameters=A22B, Type=Instruct, Version=25072026.0579.22
ERNIE-4.5Parameters=300B, Active Parameters=A47B, Type=PT2026.0578.72
ChainBackbone=Qwen-3-30B-A3B2026.0578.5
Kimi-K2Type=Instruct2026.0578.39
CliqueBackbone=Qwen-3-30B-A3B2026.0578
PEARBackbone=Claude-Haiku-4.52026.0577.5
GLM-Z1-32B-04142026.0577.42
RandomBackbone=Qwen-3-30B-A3B2026.0577.2
Claude 3.5 Sonnetshot=5-shot, CoT=true2024.0777
RingBackbone=Qwen-3-30B-A3B2026.0577
StarBackbone=Qwen-3-30B-A3B2026.0576.5
DCPOBackbone=Qwen3-4B, Optimization Method=DCPO2026.0176.1
Trinity Large Previewtuning=instruct-tuned2026.0275.25
GPT-5-nano-2025-08-07Setting=high2026.0575.02
GPT-4oshot=5-shot, CoT=true2024.0774
QwQ-32B2026.0573.94
Qwen3Parameters=30B, Active Parameters=A3B, Type=Instruct, Version=25072026.0573.93
Hunyuan-A13BType=Instruct2026.0573.84
GLM-4-32B-04142026.0573.47
Llama 3 405Bshot=5-shot, CoT=true2024.0773.3
PEARBackbone=GPT-5.4-nano2026.0573
gpt-oss-20bSetting=high2026.0572.84
Qwen3-4BMode=Thinking, Version=25072026.0572.78
Qwen3-4BBackbone=Qwen3-4B, Optimization Method=Base2026.0172.67
MiniCPM-Sala2026.0571.99
CliqueBackbone=Claude-Haiku-4.52026.0571
ERNIE-4.5Parameters=21B, Active Parameters=A3B, Mode=Thinking2026.0570.79
GPT-5-mini-2025-08-07Setting=high2026.0570.24
Qwen3-4B-InstructBackbone=Qwen3-4B-Instruct2026.0469.6
RandomBackbone=Claude-Haiku-4.52026.0569
CliqueBackbone=GPT-5.4-nano2026.0568.5
ChainBackbone=Claude-Haiku-4.52026.0568
Gemma-3-27B-it2026.0567.84
SpuriousModel=Qwen/Qwen3.5-27B2026.0567.12
RTT-AONBackbone=Qwen3-4B-Instruct2026.0466.76
GCWMModel Scale=Qwen3-8B2026.0566.5
PEARBackbone=Qwen-2.5-14B2026.0566.5
StarBackbone=Claude-Haiku-4.52026.0566.5
Llama 3 70Bshot=5-shot, CoT=true2024.0766.4
GCWMModel Scale=Qwen3-14B2026.0566.2
GPT-4oAccess=Closed-source2026.0166
RandomBackbone=GPT-5.4-nano2026.0566
Gemma 2 27BShots=5-shot2026.0265.15
ChainBackbone=GPT-5.4-nano2026.0565
TAModel Scale=Qwen3-14B2026.0564.9
GPT-4 (0125)shot=5-shot, CoT=true2024.0764.8
PEARBackbone=Gemma-3-12B2026.0564.5
CoT-SCBackbone=Qwen-3-30B-A3B2026.0564.5
RingBackbone=GPT-5.4-nano2026.0564.5
GCWMModel Scale=Qwen3-4B2026.0563.9