List your company
ResearchFormerly WizWand

General Knowledge on MMLU-Pro (Score)

82MMLU-Pro General Knowledge ScoreNemotron-3-Puzzle-75B-A9B
6.641626.205845.7765.3342Jan 6, 2026Feb 5, 2026Mar 7, 2026Apr 6, 2026May 6, 2026Jun 5, 2026Jul 5, 2026
Updated 3mo ago

Evaluation results

MethodLinks
Nemotron-3-Puzzle-75B-A9BModel=Nemotron-3-Puzzle-75B-A9B, Precision=FP82026.0782
Qwen-3-30B-A3BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.0276.7
Qwen-3-32BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.0274.1
MiMo-V2 FlashModel Variant=Base, # Shots=5-shot, # Activated Params=15B, # Total Params=309B2026.0273.2
MiMo-V2-Flash Base# Shots=5-shot, # Activated Params=15B, # Total Params=309B2026.0173.2
Qwen-3-14BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.0271.1
Llama-3.3-70BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.0270.4
Kimi-K2Model Variant=Base, # Shots=5-shot, # Activated Params=32B, # Total Params=1043B2026.0269.2
Kimi-K2 Base# Shots=5-shot, # Activated Params=32B, # Total Params=1043B2026.0169.2
Mistral-3.2-24BOpenness=Open-weights, Regional Origin=European, Post-training=Instruction-tuned2026.0267.4
Gemma-3-27BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.0266.6
OLMo-3.1-32BOpenness=Fully-open, Regional Origin=Non-European, Post-training=Instruction-tuned2026.0266.5
Step 3.5 FlashModel Variant=Base, # Shots=5-shot, # Activated Params=11B, # Total Params=196B2026.0262.3
DeepSeek V3.2Model Variant=Exp Base, # Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.0262.1
DeepSeek-V3.2 Exp Base# Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.0162.1
Gemma-3-12BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.0259.9
Mellum 2Parameters=2.5B/12B2026.0559.3
DeepSeek V3.1Model Variant=Base, # Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.0258.8
DeepSeek-V3.1 Base# Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.0158.8
OLMo-3-7BOpenness=Fully-open, Regional Origin=Non-European, Post-training=Instruction-tuned2026.0256.9
Qwen3.5-4BParameters=4B2026.0552.4
Qwen3-4BParameters=4B2026.0551.5
EuroLLM-22BOpenness=Fully-open, Regional Origin=European, Post-training=Instruction-tuned2026.0250.8
Qwen2.5-7BParameters=7B2026.0548.6
Full AttentionSetting=CPT setting2026.0446.36
Llama-3.1-8BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.0245.8
KSASetting=CPT setting2026.0445.7
Hybrid-KSASetting=CPT setting2026.0445.39
Hybrid-SWASetting=CPT setting2026.0445.23
Hybrid-SCASetting=CPT setting2026.0445.11
EuroLLM-9BOpenness=Fully-open, Regional Origin=European, Post-training=Instruction-tuned2026.0242.3
Apertus-70BOpenness=Fully-open, Regional Origin=European, Post-training=Instruction-tuned2026.0241.9
Hybrid-LinearSetting=CPT setting2026.0438.83
OLMo-3-7BParameters=7B2026.0534.5
Apertus-8BOpenness=Fully-open, Regional Origin=European, Post-training=Instruction-tuned2026.0232.7
ReMiTModel Family=SmolLM3-3B, Mid-Training=True, Few-shot=True2026.0230.73
Vanilla NTPModel Family=SmolLM3-3B, Mid-Training=True, Few-shot=True2026.0230.68
MiniPLMModel Family=SmolLM3-3B, Mid-Training=True, Few-shot=True2026.0229.52
RHO-1Model Family=SmolLM3-3B, Mid-Training=True, Few-shot=True2026.0228.19
ReMiTModel Family=Youtu-LLM-2B, Mid-Training=True, Few-shot=True2026.0225.85
MiniPLMModel Family=Youtu-LLM-2B, Mid-Training=True, Few-shot=True2026.0225.34
Vanilla NTPModel Family=Youtu-LLM-2B, Mid-Training=True, Few-shot=True2026.0224.93
Pre-TrainedModel Family=SmolLM3-3B, Mid-Training=Standard Checkpoint, Few-shot=True2026.0223.24
Hybrid-KSAsetting=Train-from-scratch2026.0422.52
KSAsetting=Train-from-scratch2026.0421.72
RHO-1Model Family=Youtu-LLM-2B, Mid-Training=True, Few-shot=True2026.0221.69
Hybrid-GDNsetting=Train-from-scratch2026.0421.22
Hybrid-SWAsetting=Train-from-scratch2026.0420.46
Hybrid-SCAsetting=Train-from-scratch2026.0420.1
Pre-TrainedModel Family=Youtu-LLM-2B, Mid-Training=Standard Checkpoint, Few-shot=True2026.0219.96
Fullsetting=Train-from-scratch2026.0419.48
ReMiTModel Family=OLMo-1B, Mid-Training=True, Few-shot=True2026.0217.44
RHO-1Model Family=OLMo-1B, Mid-Training=True, Few-shot=True2026.0213.68
Vanilla NTPModel Family=OLMo-1B, Mid-Training=True, Few-shot=True2026.0213.31
MiniPLMModel Family=OLMo-1B, Mid-Training=True, Few-shot=True2026.0213.15
Pre-TrainedModel Family=OLMo-1B, Mid-Training=Standard Checkpoint, Few-shot=True2026.029.54