General Knowledge on MMLU-Pro (Score)
82MMLU-Pro General Knowledge ScoreNemotron-3-Puzzle-75B-A9B
Evaluation Results
| Method | Links | |
|---|---|---|
| Nemotron-3-Puzzle-75B-A9BModel=Nemotron-3-Puzzle-75B-A9B, Precision=FP82026.07 | 82 | |
| Qwen-3-30B-A3BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 76.7 | |
| Qwen-3-32BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 74.1 | |
| MiMo-V2 FlashModel Variant=Base, # Shots=5-shot, # Activated Params=15B, # Total Params=309B2026.02 | 73.2 | |
| MiMo-V2-Flash Base# Shots=5-shot, # Activated Params=15B, # Total Params=309B2026.01 | 73.2 | |
| Qwen-3-14BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 71.1 | |
| Llama-3.3-70BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 70.4 | |
| Kimi-K2Model Variant=Base, # Shots=5-shot, # Activated Params=32B, # Total Params=1043B2026.02 | 69.2 | |
| Kimi-K2 Base# Shots=5-shot, # Activated Params=32B, # Total Params=1043B2026.01 | 69.2 | |
| Mistral-3.2-24BOpenness=Open-weights, Regional Origin=European, Post-training=Instruction-tuned2026.02 | 67.4 | |
| Gemma-3-27BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 66.6 | |
| OLMo-3.1-32BOpenness=Fully-open, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 66.5 | |
| Step 3.5 FlashModel Variant=Base, # Shots=5-shot, # Activated Params=11B, # Total Params=196B2026.02 | 62.3 | |
| DeepSeek V3.2Model Variant=Exp Base, # Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.02 | 62.1 | |
| DeepSeek-V3.2 Exp Base# Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.01 | 62.1 | |
| Gemma-3-12BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 59.9 | |
| Mellum 2Parameters=2.5B/12B2026.05 | 59.3 | |
| DeepSeek V3.1Model Variant=Base, # Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.02 | 58.8 | |
| DeepSeek-V3.1 Base# Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.01 | 58.8 | |
| OLMo-3-7BOpenness=Fully-open, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 56.9 | |
| Qwen3.5-4BParameters=4B2026.05 | 52.4 | |
| Qwen3-4BParameters=4B2026.05 | 51.5 | |
| EuroLLM-22BOpenness=Fully-open, Regional Origin=European, Post-training=Instruction-tuned2026.02 | 50.8 | |
| Qwen2.5-7BParameters=7B2026.05 | 48.6 | |
| Full AttentionSetting=CPT setting2026.04 | 46.36 | |
| Llama-3.1-8BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 45.8 | |
| KSASetting=CPT setting2026.04 | 45.7 | |
| Hybrid-KSASetting=CPT setting2026.04 | 45.39 | |
| Hybrid-SWASetting=CPT setting2026.04 | 45.23 | |
| Hybrid-SCASetting=CPT setting2026.04 | 45.11 | |
| EuroLLM-9BOpenness=Fully-open, Regional Origin=European, Post-training=Instruction-tuned2026.02 | 42.3 | |
| Apertus-70BOpenness=Fully-open, Regional Origin=European, Post-training=Instruction-tuned2026.02 | 41.9 | |
| Hybrid-LinearSetting=CPT setting2026.04 | 38.83 | |
| OLMo-3-7BParameters=7B2026.05 | 34.5 | |
| Apertus-8BOpenness=Fully-open, Regional Origin=European, Post-training=Instruction-tuned2026.02 | 32.7 | |
| ReMiTModel Family=SmolLM3-3B, Mid-Training=True, Few-shot=True2026.02 | 30.73 | |
| Vanilla NTPModel Family=SmolLM3-3B, Mid-Training=True, Few-shot=True2026.02 | 30.68 | |
| MiniPLMModel Family=SmolLM3-3B, Mid-Training=True, Few-shot=True2026.02 | 29.52 | |
| RHO-1Model Family=SmolLM3-3B, Mid-Training=True, Few-shot=True2026.02 | 28.19 | |
| ReMiTModel Family=Youtu-LLM-2B, Mid-Training=True, Few-shot=True2026.02 | 25.85 | |
| MiniPLMModel Family=Youtu-LLM-2B, Mid-Training=True, Few-shot=True2026.02 | 25.34 | |
| Vanilla NTPModel Family=Youtu-LLM-2B, Mid-Training=True, Few-shot=True2026.02 | 24.93 | |
| Pre-TrainedModel Family=SmolLM3-3B, Mid-Training=Standard Checkpoint, Few-shot=True2026.02 | 23.24 | |
| Hybrid-KSAsetting=Train-from-scratch2026.04 | 22.52 | |
| KSAsetting=Train-from-scratch2026.04 | 21.72 | |
| RHO-1Model Family=Youtu-LLM-2B, Mid-Training=True, Few-shot=True2026.02 | 21.69 | |
| Hybrid-GDNsetting=Train-from-scratch2026.04 | 21.22 | |
| Hybrid-SWAsetting=Train-from-scratch2026.04 | 20.46 | |
| Hybrid-SCAsetting=Train-from-scratch2026.04 | 20.1 | |
| Pre-TrainedModel Family=Youtu-LLM-2B, Mid-Training=Standard Checkpoint, Few-shot=True2026.02 | 19.96 | |
| Fullsetting=Train-from-scratch2026.04 | 19.48 | |
| ReMiTModel Family=OLMo-1B, Mid-Training=True, Few-shot=True2026.02 | 17.44 | |
| RHO-1Model Family=OLMo-1B, Mid-Training=True, Few-shot=True2026.02 | 13.68 | |
| Vanilla NTPModel Family=OLMo-1B, Mid-Training=True, Few-shot=True2026.02 | 13.31 | |
| MiniPLMModel Family=OLMo-1B, Mid-Training=True, Few-shot=True2026.02 | 13.15 | |
| Pre-TrainedModel Family=OLMo-1B, Mid-Training=Standard Checkpoint, Few-shot=True2026.02 | 9.54 |